-- Logs begin at Fri 2022-06-24 13:47:13 UTC, end at Fri 2022-06-24 15:45:46 UTC. -- Jun 24 14:22:41.413039 dv-cinder-844601-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 14:22:41.421826 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 14:22:41 2022] *** Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: nodename: dv-cinder-844601-5-iscsi Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: machine: x86_64 Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: clock source: unix Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: pcre jit disabled Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: detected number of CPU cores: 4 Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: current working directory: / Jun 24 14:22:41.424000 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: detected binary path: /usr/bin/uwsgi-core Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: your processes number limit is 47680 Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: your memory page size is 4096 bytes Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: detected max file descriptor number: 2048 Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: lock engine: pthread robust mutexes Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: thunder lock: enabled Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 14:22:41.424681 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 14:22:41.469005 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: Python main interpreter initialized at 0x5610706ed9c0 Jun 24 14:22:41.469005 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: python threads support enabled Jun 24 14:22:41.469005 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: your server socket listen backlog is limited to 100 connections Jun 24 14:22:41.469005 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: your mercy for graceful operations on workers is 90 seconds Jun 24 14:22:41.469350 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 14:22:41.469422 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: *** Operational MODE: preforking *** Jun 24 14:22:41.469686 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: *** uWSGI is running in multiple interpreter mode *** Jun 24 14:22:41.469686 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: spawned uWSGI master process (pid: 80340) Jun 24 14:22:41.469980 dv-cinder-844601-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 14:22:41.470997 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: spawned uWSGI worker 1 (pid: 80352, cores: 1) Jun 24 14:22:41.470997 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: spawned uWSGI worker 2 (pid: 80353, cores: 1) Jun 24 14:22:41.470997 dv-cinder-844601-5-iscsi devstack@keystone.service[80340]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 14:22:46.967232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:22:46.967232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jun 24 14:22:46.968034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:22:46.968034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: from cryptography.utils import int_from_bytes Jun 24 14:22:46.968458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:22:46.968458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: from cryptography.utils import int_from_bytes Jun 24 14:22:46.968560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 14:22:46.968560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: from cryptography.utils import int_from_bytes Jun 24 14:22:47.030772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 14:22:47.030953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 14:22:47.031150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] command line args: [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 14:22:47.031277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 14:22:47.031425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 14:22:47.031504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 14:22:47.031605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ================================================================================ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 14:22:47.031693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] command line args: [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 14:22:47.031863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] admin_token = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.032068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 14:22:47.032701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_dir = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.032891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ================================================================================ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 14:22:47.033027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_source = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] debug = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_dir = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] config_source = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.033932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] debug = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.034942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_config_append = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_dir = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_config_append = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_dir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.035960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.036940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.037938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.038872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] publish_errors = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] publish_errors = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.039888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] standard_threads = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.040937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] standard_threads = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_journal = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_json = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_stderr = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.041900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_journal = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_syslog = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_json = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_stderr = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] use_syslog = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.042648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 14:22:47.042729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.042842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.042995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.043984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.044952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.045994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.046971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.047796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.048960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.049910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.050872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.051972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.052954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.053871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.054987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.055919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.056982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.057887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.058861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.059983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.060930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.061955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.062927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.063932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.064881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.065948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.066925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.external = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.external = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.067970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.password = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.password = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.068946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.token = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] auth.token = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.069875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.070992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.071958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.072947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.073946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.074941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.075972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.076922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.077886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.078783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.079998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.080942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.081984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.082924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.083922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.084860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.085975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.086995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.087930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.088865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.089966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.090940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.091980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.092970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.093895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.094060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.094138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.094302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.094382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.095971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.096970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.097852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.098986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.099921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.100970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.101936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.102894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.103876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.104987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.105962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.106933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.107885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.108995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.109958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.110948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.111938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.112904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.113860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.114989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.115873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.116962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.117974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.118942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 14:22:47.119795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 14:22:47.119949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80353) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 14:22:47.150577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.150732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.150811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.150906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.151023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.151109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.160199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.160395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.188762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.188990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.189913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.190142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.332236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.333778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.381704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.381940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.384949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.385191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.446225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.446448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.446697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.446782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.446916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.447780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.472510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.472735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.472922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.473875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.474005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.474292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.484763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.485277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.497548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.497781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80353) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.498988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.499186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.499412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.499630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.499846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 14:22:47.517746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.517947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:22:47.518029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:22:47.518280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.518376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.518479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:22:47.518580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 14:22:47.518779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.518860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.519620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.530294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.530569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.530652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.530842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.530925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.531116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.531211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.531429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.539469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.539949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.542758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.543033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.543294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.543376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.543570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.543650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.543843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.543924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.544957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.545931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.572642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.573848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.574243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.574541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.575505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.575784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.580385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.580770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.581046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.581757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.582151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.582471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.588317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.588592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.589905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.590189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.593067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.593345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.593620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.593897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.594166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.594482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.594776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.594847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.595036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.595111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.595444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.595668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.595946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.596221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.596490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.596759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.613916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.614183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.615939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.616217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.618897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.620905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.622505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.623152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.624601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.625000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.629355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.630782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.633216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.634428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.637181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.637472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.637749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.638977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.639253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.639530 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.658458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.658753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.659014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.659281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.659457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.659743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.660011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.660284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.667178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.668224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.668873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.669158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.669946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.670222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.672354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.672742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.673023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.673300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.673543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.673628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.673842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.674142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.674447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.674816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.687089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.687363 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.687644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.687920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.688992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.689275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.689557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.689837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.690040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.690338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.690656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.691513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.691803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.692093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.709065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.710806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.712365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.714128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.715687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.717402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.717483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.719064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.719206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.719337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.719691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.720924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.721203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.721482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.727045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.728632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.728901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.730541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.731901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.732186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.732568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.732841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.733813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.734105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.734495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.734796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.742623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.742905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.743186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.744523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.744811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.745092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.751861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80353) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.752132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.752405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.752674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.752939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.753313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.753590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.753671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 14:22:47.753856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.753937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.754947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80353) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.755914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.756184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.756455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.756726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.757005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.757275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 14:22:47.793119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610706ed9c0 pid: 80353 (default app) Jun 24 14:22:47.795318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610706ed9c0 pid: 80352 (default app) Jun 24 14:22:47.796526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a54c8-b4cb-4c6b-91c1-492b6b15340f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:47.796814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a54c8-b4cb-4c6b-91c1-492b6b15340f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:47.797090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a54c8-b4cb-4c6b-91c1-492b6b15340f None None] PATH_INFO: `/v3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:47.801945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Fri Jun 24 14:22:47 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:22:56.352934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:56.353529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:56.353529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:56.357887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:22:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:22:56.363867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:56.364286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:56.364426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:56.434434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: INFO dbcounter [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] Registered counter for database keystone Jun 24 14:22:56.456422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG oslo_db.sqlalchemy.engines [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80353) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 14:22:56.462612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writer thread running {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 14:22:56.639868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80353) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 14:22:56.640200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG passlib.handlers.bcrypt [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80353) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 14:22:56.695139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:22:56.696740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:22:56.818987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:22:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 14:22:56.982678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:22:57.052411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: INFO dbcounter [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] Registered counter for database keystone Jun 24 14:22:57.071517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG oslo_db.sqlalchemy.engines [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 14:22:57.077667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writer thread running {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 14:22:57.198736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1b8c0-aec0-48a6-b841-24daeecbebf2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:22:57.202978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:22:57.207420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:57.207420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:57.207420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:57.212484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:22:57.265380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 14:22:57.265685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca5629-3e1b-4af9-af6f-d81034269b7b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:22:57.275236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Jun 24 14:22:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:22:57.281761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:22:57.296067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bb8c7-43e9-4b89-9389-70ae8029c51a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:22:57.301109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:22:57.302216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:57.302502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:57.302813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] PATH_INFO: `/v3/projects/7486fcd7991c440786c9d72536ea0aed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:57.306835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] RBAC: Authorizing `identity:get_project(project_id=7486fcd7991c440786c9d72536ea0aed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:22:57.358053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 14:22:57.358392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774286b-5a9c-4228-a8f9-eb07ae2ff5e3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:22:57.363897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Fri Jun 24 14:22:57 2022] GET /identity/v3/projects/7486fcd7991c440786c9d72536ea0aed => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:22:59.372620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8fe44-9427-4690-8097-0ae4b7fec944 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:59.373055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8fe44-9427-4690-8097-0ae4b7fec944 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:59.373175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8fe44-9427-4690-8097-0ae4b7fec944 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:59.377552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:22:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:22:59.383023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:59.383487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:59.383594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:59.402056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:22:59.403640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:22:59.412150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:22:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 14:22:59.434636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8fe44-9427-4690-8097-0ae4b7fec944 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:22:59.446599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8fe44-9427-4690-8097-0ae4b7fec944 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:22:59.450789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:22:59.451422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:59.451704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:59.451979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:59.459608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:22:59.461375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6319e1-8440-4678-978e-36579472f1b0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:22:59.470504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Fri Jun 24 14:22:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:22:59.475704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:22:59.485149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd2377-e28f-499f-8bb0-173af53ffaf1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:22:59.489539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:22:59.490384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:22:59.490702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:22:59.490979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:22:59.492353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:22:59.494172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d77f9-725c-4b88-8dd5-7010bec7a46e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:22:59.495755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Fri Jun 24 14:22:59 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:01.591257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36c4e-d01c-4ecd-ae8e-6ae06df6bbf2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:01.591779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36c4e-d01c-4ecd-ae8e-6ae06df6bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:01.591862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e36c4e-d01c-4ecd-ae8e-6ae06df6bbf2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:01.593172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:01.599112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:01.599987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:01.600587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:01.617500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:01.618651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:01.676829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7c287-dfb2-492e-a90b-1609fbef5b21 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:01.677110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7c287-dfb2-492e-a90b-1609fbef5b21 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:01.677394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7c287-dfb2-492e-a90b-1609fbef5b21 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:01.678782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:01.683993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:01.684279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:01.684558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:01.699347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:01.722668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:01.724320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG passlib.handlers.bcrypt [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80352) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 14:23:01.724749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG passlib.handlers.bcrypt [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80352) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 14:23:01.745898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:01.770141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:01.777042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:01.826198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbd1f0-b3bb-4316-9871-99a60c30c956 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:01.859931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:01.860724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:01.861008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:01.861282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:01.872275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:01.873154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:01.883617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:01.884699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:01.885716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:01.890986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:01.895294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:01.979715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:01.993244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9329f1-3228-4e21-aa52-830fbe91189b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:01.995670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:01.998142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59727c3e-7d0c-4e81-b0c7-94544d6061b5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.031926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.032848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.033133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.033402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.034327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:02.034420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.034677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.035222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.035651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.035937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.036204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.048598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:02.054072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:02.103523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.105024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.154553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda718d3-74e1-4705-8802-140f4104c4f0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.184558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:02.188262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.189133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.189412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.189683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.191258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.192713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.194550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:02 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:02.201037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:02.203437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 8/17] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:23:02.209051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:02.234551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:02.299765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f66e6-fc87-4e88-955f-a22c67fcb0b5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.307834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0092d85-fce1-45a1-92ca-4fe47af49427 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.333423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.334327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.334705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.335218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.336985 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.338639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.340717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:02.342872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.344428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.344878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.345158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.348104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:02.355957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.357456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:23:02.377959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:02.378818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:02.379637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:23:02.379637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:23:02.379637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:23:02.379637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:02.379637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:02.384713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:02.419545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:02.451762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f3500e-3160-4b6e-8e83-eb28f41110fb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.453967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:02.484443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8970d8-c192-4370-af4c-5865d8cc5a9e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:02.486196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.486887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.487174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.487450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.488328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.domain.id=default, user.id=ecb45340b1984b23904f9e0f1752bcc6) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:02.488639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.490102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec880e83-4528-411a-ab3b-e743830ccd36 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.498614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Jun 24 14:23:02 2022] GET /identity/v3/role_assignments?user.id=ecb45340b1984b23904f9e0f1752bcc6&scope.domain.id=default&role.id=4ef72abf5489405bbc82b2318479765b => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:02.519157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:02.519848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:02.520135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:02.520415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:02.521165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:02.521477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:02.522957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89192cf9-e5a3-4747-99ac-3fbb13128e52 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:02.543231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:04.751471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a4c0-024a-443d-8077-a5c18828edfc None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:04.751471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a4c0-024a-443d-8077-a5c18828edfc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:04.751471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a4c0-024a-443d-8077-a5c18828edfc None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:04.751471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:04.764197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:04.764675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:04.764746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:04.783543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:04.785080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:04.866802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:04.901401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898a4c0-024a-443d-8077-a5c18828edfc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:05.056170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898a4c0-024a-443d-8077-a5c18828edfc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:05.107236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:05.108044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.108330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.108611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.122340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-355f2b87-0d90-4245-a56b-1c96facc2889 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:05.134146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:05.135095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:05.135862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:05.139723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 13/24] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:05.146677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:05.198128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:05.295359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14aaa6-4477-4235-b73b-30130add9601 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:05.342030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:05.343056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.344135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.344617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.345568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:05.346080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:05.347688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:05.360471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 12/25] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:05.368665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2b43-b0e7-41c6-8c3b-679284c12ace None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.368958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2b43-b0e7-41c6-8c3b-679284c12ace None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.369234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a2b43-b0e7-41c6-8c3b-679284c12ace None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.373967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:05.375736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:05.378097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3029-4f8c-42e3-9d7a-1c6525f301f4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.379754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3029-4f8c-42e3-9d7a-1c6525f301f4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.380254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f3029-4f8c-42e3-9d7a-1c6525f301f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.424386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-030f3029-4f8c-42e3-9d7a-1c6525f301f4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:05.427022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-030f3029-4f8c-42e3-9d7a-1c6525f301f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:05.477050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:05.536497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:05.567517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9fb87-618a-4c9e-862b-8a9380fe9510 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:05.579886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 15/27] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:05.581903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d2aff-1991-4243-829f-4c12737038d0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.582196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d2aff-1991-4243-829f-4c12737038d0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.582474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d2aff-1991-4243-829f-4c12737038d0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.584322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 16/28] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:05.585664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63399-7740-461c-9364-bf47328ceefc None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.585952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63399-7740-461c-9364-bf47328ceefc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.586231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f63399-7740-461c-9364-bf47328ceefc None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.587978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 17/29] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:05.595135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.595911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.596199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.620248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:05.621145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.621350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:05.621473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.621687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.622654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:05.622956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:05.624968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac310a-644e-4f6d-88b8-04ac51aa0202 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:05.627809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 13/30] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:05 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:05.629022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.629309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.629588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.657765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:05.659350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:05.703801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 18/31] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:05.704947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:05.748307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 14/32] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:05.749151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:05.816373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:05.831462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8207f2-ca0f-4360-aca5-bc097dfb4bcf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:05.873183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:05.875341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:05.876013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.876291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.876560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.877299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:05.878715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:05.895057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a730c-c7af-43f3-ba75-ba8e1fb6e0c7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:05.930458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:05.931294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:05.931576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:05.931849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:05.933322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:05.934809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:05.941058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 15/33] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:05.941058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.049265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9f85c-35ae-478f-b272-e3aadc460df8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:06.087685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:06.088514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.088798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.101085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:06.102696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:23:06.122220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:06.123307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:06.124129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:23:06.124129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:23:06.124129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:23:06.124129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:06.124129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 16/34] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:06.124455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.131981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:06.140863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 19/35] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Jun 24 14:23:05 2022] POST /identity/v3/roles => generated 219 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:06.142237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.254307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7895e6f1-2a00-4cbd-9a12-3677888499aa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:06.271198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5d5e1-7845-4010-80b2-8ae26e888272 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:06.295216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:06.296987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.297482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.297953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.298971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:06.300724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd4504-9d9a-4ea9-8f1c-3c106604a20c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:06.316051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:06.316939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.327991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.327991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] PATH_INFO: `/v3/domains/default/users/ecb45340b1984b23904f9e0f1752bcc6/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.327991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ecb45340b1984b23904f9e0f1752bcc6, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:06.327991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8985cf6-9538-4219-9c4b-d7b32dedf209 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:06.409054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 20/36] 10.222.0.23 () {58 vars in 1361 bytes} [Fri Jun 24 14:23:06 2022] PUT /identity/v3/domains/default/users/ecb45340b1984b23904f9e0f1752bcc6/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:06.410133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e05785-4e72-4c9f-8297-98ab39d379d2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.410413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e05785-4e72-4c9f-8297-98ab39d379d2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.410721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e05785-4e72-4c9f-8297-98ab39d379d2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.414695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 21/37] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:06.414695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e05785-4e72-4c9f-8297-98ab39d379d2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.432243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:06.496361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 17/38] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Jun 24 14:23:06 2022] POST /identity/v3/roles => generated 225 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:06.500409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366468b-3a4e-4c33-9fa6-3bf5c170c701 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.501903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366468b-3a4e-4c33-9fa6-3bf5c170c701 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.502194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366468b-3a4e-4c33-9fa6-3bf5c170c701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.559101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e05785-4e72-4c9f-8297-98ab39d379d2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:06.612930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:06.613633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.613917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.614242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.615053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:06.615363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:06.616976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda0b1e-de7c-4e30-8ed8-fde733ed17a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:06.643980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 22/39] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:06.645518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d490e-32f6-4303-994e-48ce3cb1139e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.645819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d490e-32f6-4303-994e-48ce3cb1139e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.646100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d490e-32f6-4303-994e-48ce3cb1139e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.647572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 23/40] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:06.652664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d490e-32f6-4303-994e-48ce3cb1139e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.659635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0366468b-3a4e-4c33-9fa6-3bf5c170c701 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:06.661285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0366468b-3a4e-4c33-9fa6-3bf5c170c701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:06.743625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:06.746122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:06.762594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 18/41] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:06.764008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.764308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.764591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.787448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:06.788641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:06.789693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d490e-32f6-4303-994e-48ce3cb1139e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:06.828997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:06.830384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:06.830384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:06.830384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:06.831205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:06.832767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:06.863783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 19/42] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:06.870567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.927142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 24/43] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:23:06 2022] POST /identity/v3/projects => generated 301 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:06.928281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:06.990920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bb279-26e9-4599-9637-13e2993f8d93 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:07.031047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:07.046669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:07.047284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:07.048163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:07.048644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:07.050558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:07.052201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:07.067836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca770-3a9e-4a90-8b95-10d39f56f8ab None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:07.088896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:07.104948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 20/44] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Jun 24 14:23:06 2022] POST /identity/v3/roles => generated 223 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:07.110262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:07.111245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:07.111602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:07.111887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:07.113238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:07.114897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad57b7-1b74-4676-a073-31697ffa8aea None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:07.117356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 25/45] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:07.122966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:07.246067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115725-e10d-40ac-a83e-8a95776ebb12 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:07.283709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:07.284385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:07.285029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:07.285029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:07.285666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:07.287178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62840d3-f970-4de6-94aa-cff58ffa1ec7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:07.374836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 21/46] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:23:07 2022] POST /identity/v3/projects => generated 312 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:23:08.508760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ced8-004a-4d2d-88cb-87a070a56ece None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:08.509296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ced8-004a-4d2d-88cb-87a070a56ece None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:08.509385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ced8-004a-4d2d-88cb-87a070a56ece None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:08.510814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 26/47] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:08.516394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:08.516810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:08.516991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:08.533987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:08.535251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:08.564476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:08.601354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 22/48] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:08.625118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2ced8-004a-4d2d-88cb-87a070a56ece None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:08.723075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2ced8-004a-4d2d-88cb-87a070a56ece None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:08.757330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:08.758122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:08.758406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:08.758710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:08.769239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:08.776956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:08.777788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:08.778651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 27/49] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:08.785067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:08.881048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb63e8-7b33-4536-acec-67d8b37f1b35 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:08.915773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:08.916438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:08.916726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:08.917005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:08.917760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:08.918069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:08.919490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:08.927616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 23/50] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:08.933296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:09.085923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:09.138456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81337f1-3d28-4304-8a65-f12dc4867d73 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:09.171792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:09.172664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.172946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.173222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.174418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:09.175914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:09.177552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 28/51] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:08 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:09.183316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:09.210920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:09.284431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52838-4cf2-4e21-8121-13a9c76ed73f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:09.317456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:09.318250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.318553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.318839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.320002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:09.321442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9ffee-0afd-490e-8729-6f674cd9fa34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:09.323090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 24/52] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:09.328881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:09.403396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a46-b62b-4bee-ba6d-5d38b8232d4f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.403617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a46-b62b-4bee-ba6d-5d38b8232d4f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.403903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a46-b62b-4bee-ba6d-5d38b8232d4f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.405816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 25/53] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:09.410908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.411196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.411502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.428022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:09.429222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:09.432138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1af4-257b-492d-8f71-48b0354d78a5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:09.446116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:09.465776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:09.466430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.466742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.467022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.468464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.domain.id=default, user.id=ecb45340b1984b23904f9e0f1752bcc6) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:09.468821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:09.470360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:09.482737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 29/54] 10.222.0.23 () {56 vars in 1322 bytes} [Fri Jun 24 14:23:09 2022] GET /identity/v3/role_assignments?user.id=ecb45340b1984b23904f9e0f1752bcc6&scope.domain.id=default&role.id=4ef72abf5489405bbc82b2318479765b => generated 530 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:09.497868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 26/55] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:09.523917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:09.629311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388308c3-f92c-4a0e-8282-0c9d36656c6b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:09.668525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:09.669252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.669537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.669815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.671714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:09.673204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f252565-129d-4537-b47e-4e45a228bd27 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:09.674880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 30/56] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:09.680475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:09.708116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:09.778627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cddaa9d-45ff-40dc-abe6-650cdea2a680 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:09.812033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:09.812680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:09.812989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:09.813269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:09.813979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:09.815439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7969-ca9e-48ff-ab92-d6f0303ac7ea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:09.906053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 27/57] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Jun 24 14:23:09 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:11.917748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7219e-2548-44ee-bedd-f8daed1d915b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:11.918257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7219e-2548-44ee-bedd-f8daed1d915b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:11.918341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7219e-2548-44ee-bedd-f8daed1d915b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:11.919693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 31/58] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:11.925128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:11.925524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:11.925695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:11.942339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:11.943554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:12.008115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 28/59] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:12.032308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7219e-2548-44ee-bedd-f8daed1d915b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:12.085950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:12.139474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7219e-2548-44ee-bedd-f8daed1d915b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:12.172815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:12.173533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:12.173817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:12.174091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:12.175283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:12.176725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ac099-bf17-47be-b1d1-9846a158fbc3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:12.178326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 32/60] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:12.184043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:12.194066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:12.279218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bde14e-c16c-493b-9190-acd56b3adf82 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:12.312339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:12.312995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:12.313274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:12.313548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:12.314247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:12.315722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f5eb6-1710-4d47-9760-81bd46eca5bd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:12.482967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 29/61] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:12 2022] POST /identity/v3/users => generated 275 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:15.538144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2c77c-b084-4c74-a8fd-c0ad6625fd2c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:15.539053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2c77c-b084-4c74-a8fd-c0ad6625fd2c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:15.539482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2c77c-b084-4c74-a8fd-c0ad6625fd2c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:15.541016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 33/62] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:15.546110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:15.547682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:15.548104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:15.571148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:15.572574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:15.649296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:15.661780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 30/63] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:15.706847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2c77c-b084-4c74-a8fd-c0ad6625fd2c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:15.856193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2c77c-b084-4c74-a8fd-c0ad6625fd2c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:15.876040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:15.903353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:15.904142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:15.904424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:15.904699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:15.919390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1ad10992-b1ff-49e3-88f6-7d944fd70d75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:15.927683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:15.928527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:15.929303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:15.932438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 34/64] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:15.938778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:15.944739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a51-9bbc-4521-b47b-cfddcbaa1621 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:15.945036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a51-9bbc-4521-b47b-cfddcbaa1621 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:15.945320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e73a51-9bbc-4521-b47b-cfddcbaa1621 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:15.946947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 35/65] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:15.951882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:15.952172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:15.952491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:15.987567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:15.989124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:16.058752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38b667-fc01-4dfd-aea7-c3c2e796a8b2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.107221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.107922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.108211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.108494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.110445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:16.110788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:16.112295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:16.123071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 36/66] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:16.126713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 31/67] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:16.132280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.152898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.234995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.264919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891e105-b91a-4afd-972d-a7e2dac07071 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.269688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd07f7b-2570-4dc8-ada2-2f10dd2331c7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.308766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.309648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.309930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.310207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.325587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.326385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.326597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:16.326715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.326977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.328706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c8abb6-f826-4b06-8288-6b6ef81543ba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:16.330422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 37/68] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:16.331645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3c13-c458-4e76-ad93-6a50eebf019b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.331931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3c13-c458-4e76-ad93-6a50eebf019b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.332207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf3c13-c458-4e76-ad93-6a50eebf019b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.333434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 38/69] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:16.334484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdeb3-5b1b-49d5-9f92-588c07cbc32e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.334789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdeb3-5b1b-49d5-9f92-588c07cbc32e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.335169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdeb3-5b1b-49d5-9f92-588c07cbc32e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.336371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 39/70] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:16.339253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:16.340964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984c701-ef18-4872-94b0-4c2762286a5d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.341257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984c701-ef18-4872-94b0-4c2762286a5d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.341541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984c701-ef18-4872-94b0-4c2762286a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:16.346917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:16.347767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:16.348531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 32/71] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:16.349155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.361873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d984c701-ef18-4872-94b0-4c2762286a5d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:16.363359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d984c701-ef18-4872-94b0-4c2762286a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:16.440341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 40/72] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:16.441695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.441995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.442275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.460176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:16.461382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:16.464077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380943d-375d-469c-861b-d52e161262b8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.487837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.507100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.507753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.508042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.508320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.528796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 41/73] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:16.537826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.559402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.586770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:16.588290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:16.637874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c345fa-95ae-4f5c-8a97-bf7e9fbf1c3f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.671580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.672282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.672565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.672846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.673600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:16.673911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:16.675347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0967898-6f56-4767-b4a9-ec4fc77bf81f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:16.683678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 42/74] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:16.684878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc36be0-361a-46db-8a8d-1b861c765f06 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.685168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc36be0-361a-46db-8a8d-1b861c765f06 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.685456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc36be0-361a-46db-8a8d-1b861c765f06 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.686714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 43/75] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:16.687464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc36be0-361a-46db-8a8d-1b861c765f06 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.695792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 33/76] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:16.696654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.714129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.773785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.789844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc36be0-361a-46db-8a8d-1b861c765f06 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.793902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fe423-7a01-4c0e-844e-5d86b79fe7c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.824855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.825654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.825936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.826210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.828422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.829214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.829496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.829772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.836154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:16.839706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:16.844386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:16.845220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:16.845986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 44/77] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:16.847349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6edf7612-da03-431d-8223-d0a64e45e40f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:16.847727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:16.848550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 34/78] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:16.854130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.854852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.854852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.868519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:16.869734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:16.935058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 35/79] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:16.935876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:16.949917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:16.950042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc3df7-cdad-4b5a-9b2c-f49061ed6336 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:16.984711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:16.985701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:16.985984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:16.986254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:16.987485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:16.988946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:16.990589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 45/80] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:16.991342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.006462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.045538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3efbe6-fe11-45f7-87d5-eb9827610e77 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.079789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.080645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.080926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.081201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.082371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.083876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.085759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 36/81] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.086427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.093956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dd630-a0c8-4a1f-914f-f0441057fa2f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.127606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.128271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.128552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.128827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.129569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:17.129875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.131285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.139244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 46/82] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.140630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.159265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.183787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9d44-6413-47a8-96a3-c4444dc3152c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.216453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.218959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.219630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.219915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.220193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.220942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:17.221250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.222656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.230828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 37/83] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.231611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.244801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c94da-770d-446a-a2ae-9e9f75b9809d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.278150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.279128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.279426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.279683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.280916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.282411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.284053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 47/84] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.284805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.331042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b5cf-a795-410c-9bd4-b7f2f238d8d9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.365013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.365678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.365966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.366247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.369776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.385200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385844b4-083d-404f-bca6-3aa5e3d93396 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.418728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.419388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.419672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.419954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.420815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=59184329dff84fd99b155057248ab763) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:17.421184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.422641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.428199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 48/85] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=66ef983ebbae442b8cab404599a2d333 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.428974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.434714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.436350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.440538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.550466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba71b84-a5fe-4704-b189-fa8795b050cc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.553786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 38/86] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:17.554453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.594501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.600233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.601637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.601963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.602240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.607698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.607698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.607849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 49/87] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.612472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.671840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7fa54-71fe-4582-bc3c-299855aa211f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.689977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.709526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.710437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.710753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.711034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.712242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.713710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.720537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 39/88] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.720673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1eba-0bee-4fa8-b628-cdb69cbdf62c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.720749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.753758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.754446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.754777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.755055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.755788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.757269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.815100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 50/89] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:17 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:17.816342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.847105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952447-39b5-4716-811e-40c57a5b34f8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:17.850888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:17.897346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:17.898639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:17.899133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:17.899628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:17.901195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:17.903028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:17.905143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 40/90] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:17.911756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:17.963111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93afebe-7506-43fd-8a8d-308a2cdcd39d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.008945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:18.014093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.014892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.015716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.016080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.040158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa6cde-ef7f-4760-b130-1b49f0c44190 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.085008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.085670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.085961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.086233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.108145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.110780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.154424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:18.189739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.191373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.238238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 51/91] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:18.240345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:18.277110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:18.307380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 41/92] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:18.308443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:18.361083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b45a8-80c2-4a45-8335-b3a319ded86b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.400676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.401350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.401633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.401912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.402946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=ecb45340b1984b23904f9e0f1752bcc6) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:18.403266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.404893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.411384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 52/93] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:18 2022] GET /identity/v3/role_assignments?user.id=ecb45340b1984b23904f9e0f1752bcc6&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:18.412195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:18.468782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:18.562151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596895a-9a9e-411b-9e93-8871393162f2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.607838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.609583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.610038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.610531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.612057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.613716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.616089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 42/94] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:18 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:18.623774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:18.654005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:18.669107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886c14-b138-4fdd-8957-0b3878bd1381 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.728980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.730145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.730643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.731119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] PATH_INFO: `/v3/projects/484d9ff52f224bb28e22bba11d369914` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.733041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] RBAC: Authorizing `identity:get_project(project_id=484d9ff52f224bb28e22bba11d369914)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.735338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.737253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 53/95] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:18 2022] GET /identity/v3/projects/484d9ff52f224bb28e22bba11d369914 => generated 312 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:18.743085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:18.818101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916d333-52dd-4363-a8a2-76e700a8117f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.877049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.878092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.878570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.879015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.880047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=59184329dff84fd99b155057248ab763) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:18.880533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.882161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1e95a-a72b-4d45-9238-45abf49cda88 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.890282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 43/96] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:18 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=a2ae414a10a54833a1002750699b5d83 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:18.921883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089647-7304-4125-8cc3-f7868ffc3237 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:18.963671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:18.964618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:18.965071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:18.965518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:18.966575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=484d9ff52f224bb28e22bba11d369914, user.id=59184329dff84fd99b155057248ab763) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:18.967128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:18.969451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cc1b6-3543-4f0a-bdbc-f1b201ad7729 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:18.976193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 54/97] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:18 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=484d9ff52f224bb28e22bba11d369914&role.id=66ef983ebbae442b8cab404599a2d333 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:20.170880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2ee4-b84e-4d4e-a5be-928ba6a5ddec None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:20.172305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2ee4-b84e-4d4e-a5be-928ba6a5ddec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:20.172755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a2ee4-b84e-4d4e-a5be-928ba6a5ddec None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:20.174298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 44/98] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:20.183902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:20.184654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:20.185134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:20.211457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:20.212934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:20.235113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:20.293015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 55/99] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:20.327706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2ee4-b84e-4d4e-a5be-928ba6a5ddec None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:20.332773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:20.447475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a2ee4-b84e-4d4e-a5be-928ba6a5ddec None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:20.483622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7323-7355-412a-bed2-818374c7742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:20.484788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7323-7355-412a-bed2-818374c7742b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:20.485288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7323-7355-412a-bed2-818374c7742b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:20.485756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c7323-7355-412a-bed2-818374c7742b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:20.496800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c7323-7355-412a-bed2-818374c7742b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-072c7323-7355-412a-bed2-818374c7742b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:20.505033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:20.505872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:20.506643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:20.509342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 45/100] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:20.515636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:20.635316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a99d82-908a-4dd7-bd31-9ea2f6f07577 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:20.683549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:20.684598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:20.685098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:20.685575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:20.686595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:20.687585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:20.689331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:20.702547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 56/101] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:20.708337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7323-7355-412a-bed2-818374c7742b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:20.775935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:20.816497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c7323-7355-412a-bed2-818374c7742b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:20.851933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:20.853164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:20.853642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:20.854109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:20.855594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:20.857341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:20.859430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 46/102] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:20 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:20.867569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:20.891746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:20.992072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a16ddb-15e0-49ae-a850-635507d15690 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.035227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.036107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.036387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.036658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.037880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:21.039369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26842b-8a7a-4aec-8463-ccc681027ab8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:21.041075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 57/103] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:20 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:21.046941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.137438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc44f0-bcd9-4368-9d5e-f7d1b8b9b081 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.137750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc44f0-bcd9-4368-9d5e-f7d1b8b9b081 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.137994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc44f0-bcd9-4368-9d5e-f7d1b8b9b081 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.140569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 58/104] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:21.148385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd571de-1635-4be0-85a0-8cedc3e43586 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.148800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd571de-1635-4be0-85a0-8cedc3e43586 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.149178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd571de-1635-4be0-85a0-8cedc3e43586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.173584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2383f28-7241-4161-aa56-bc5ee6afcd2c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.188984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bdd571de-1635-4be0-85a0-8cedc3e43586 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:21.190873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bdd571de-1635-4be0-85a0-8cedc3e43586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:21.238369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:21.244863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.246165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.247426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.251905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/59184329dff84fd99b155057248ab763/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.253930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, user_id=59184329dff84fd99b155057248ab763, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:21.255765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a537af2-279e-466e-8df7-ed43b8a7c69d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:21.307405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 59/105] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:21.308935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4907f2-814d-475d-a7b9-78ef6fb5adc6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.309376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4907f2-814d-475d-a7b9-78ef6fb5adc6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.309773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4907f2-814d-475d-a7b9-78ef6fb5adc6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.311236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 60/106] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:21.312415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b1deb-536c-4c1d-b035-278700b2ddb0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.312834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b1deb-536c-4c1d-b035-278700b2ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.313265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b1deb-536c-4c1d-b035-278700b2ddb0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.314639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 61/107] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:21.322646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 47/108] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:21 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/59184329dff84fd99b155057248ab763/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:21.324023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.324524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.324951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.345822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.346434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.346882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.356500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:21.360643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:21.376054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:21.378385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:21.385274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:21.511945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 62/109] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:21.512776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.525958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 48/110] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:21.544995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.650624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb57c81-9f9c-47c7-809d-52384dd5a9d5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.673236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b2332-5458-476f-ac45-223665c3192c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.697432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:21.699795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.700696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.700942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.701223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.707348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:21.708926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:21.710725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 63/111] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:21.711722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.714353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.715485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.715963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.716426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.732405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:21.745925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:21.754291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:21.757934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 49/112] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:21.758786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.849858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c5ff2-cedd-4e6a-a04e-42c97422b403 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.895121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.895920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.896198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.896362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49529f-683f-455a-bf13-322a1d601269 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:21.896455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.909357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:21.918994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:21.919819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:21.920586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:21.921694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 64/113] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:21.927247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:21.938958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:21.939639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:21.940414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:21.940709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:21.941861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:21.943674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0edb60-3191-435a-92ba-469daf1a66a5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.008920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.033370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 50/114] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:23:21 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:22.035215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787605d4-23f4-4992-9df3-6e4854da9403 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.035703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787605d4-23f4-4992-9df3-6e4854da9403 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.036225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787605d4-23f4-4992-9df3-6e4854da9403 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.038000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 51/115] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:22.039593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787605d4-23f4-4992-9df3-6e4854da9403 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.047643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef33f4-7a08-40be-913c-0847b5ca122c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.115018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.130459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.131366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.131751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.132127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.133254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:22.133683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.135281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52079a89-c9a6-41d2-9747-9dd0e1b8c025 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.148149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 65/116] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:22.149560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.149943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.150303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.169706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:22.171909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:22.273935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787605d4-23f4-4992-9df3-6e4854da9403 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.279434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 66/117] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:22.281537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.321155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.322143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.322630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.323074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.324031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:22.324509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.326079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f256f-3839-4da1-ba92-9af0af7dcc4d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.338804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 52/118] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:22.338804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d435-070d-4b0b-a3e5-561ac0f13b58 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.339041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d435-070d-4b0b-a3e5-561ac0f13b58 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.339367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d435-070d-4b0b-a3e5-561ac0f13b58 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.341045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 53/119] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:22.341816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d435-070d-4b0b-a3e5-561ac0f13b58 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.373374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.417143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af644d6-dba1-49b7-b495-9b1992a2ad48 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.459571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.460709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.461142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.461557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] PATH_INFO: `/v3/users/a087e0fc3b104c7bb645f80189e77c24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.475846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] RBAC: Authorizing `identity:get_user(user_id=a087e0fc3b104c7bb645f80189e77c24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.477590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.479604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 67/120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/users/a087e0fc3b104c7bb645f80189e77c24 => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:22.479983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.481333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.488699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d435-070d-4b0b-a3e5-561ac0f13b58 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.537720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.538648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.538985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.539239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.550342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-604b1d44-5eac-48b4-a08e-9680152c39eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:22.559855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:22.560818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:22.562948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 54/121] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:22.563618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.563618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.563889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.582747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:22.583588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:22.615919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1fe72e-19c7-4652-b568-1adcde1c77e3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.647724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.658699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.660979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.660979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.660979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.661519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.661758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 55/122] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:22.666235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.667339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.671579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 68/123] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:22.671579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.779452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.787881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5308e2-aef5-4f7b-90f8-2596edc05663 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.796690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745008e3-e4ab-4b60-9935-33cd1a63ff03 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:22.828335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.829247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.829361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.829667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.830426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:22.830764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.832161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.840941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 69/124] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:22.841560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:22.843129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:22.843953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:22.844614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:22.846036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:22.905031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:22.926891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:22.928393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:22.963316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50522b9c-160b-409b-befa-b20fd81d81b6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.007841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.009285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.009741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.010467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.012105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.014257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.017292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 70/125] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.017559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.022562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.038978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 56/126] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:23.040001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.152831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacca958-fc3c-4a7f-9733-3f42dcb3642a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.155166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c512df5-5862-48a0-a04f-614c1737f60c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.189720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.190089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.191243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.191719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.192188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.193619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.195584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.197616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 57/127] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.198505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.204557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.205372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.205659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.205941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.222559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:23.233885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:23.234812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:23.235598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 71/128] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:23.236241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.262934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.310963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2738e9-03a7-4dd0-b5c6-46c6923240c7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.359208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.360466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.360969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.361435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/ecb45340b1984b23904f9e0f1752bcc6/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.368723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, user_id=ecb45340b1984b23904f9e0f1752bcc6, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.370407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.385090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db9131-7f2c-41d8-a5b0-987c404116f8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.421932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.422744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.423024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.423344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.424532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.425991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.427966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 72/129] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.428638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.480506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.485052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 58/130] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:23 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/ecb45340b1984b23904f9e0f1752bcc6/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:23.486354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.567063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f64f72-ecde-49bb-8ac6-6016abd8de4b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.616024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f16f3-cdce-46ae-acc7-31af10eb423e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.634723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.643039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.644251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.644670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.645128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.648646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.650317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.652278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 73/131] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.653179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.660323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.663311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.663311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.663311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.663311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:23.663311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.666016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.677084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 59/132] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.677628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.802842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737644c-8886-43e3-bb8a-f33c51afcd0b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.815962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758a429-8c77-471d-a455-f50912cac16d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:23.829531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:23.851078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.851747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.852053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.852356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.852720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:23.853200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=a087e0fc3b104c7bb645f80189e77c24) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:23.853524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.853623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.853891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.854171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/59184329dff84fd99b155057248ab763/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.855027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.856471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, user_id=59184329dff84fd99b155057248ab763, role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:23.858308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af8771-6492-4970-8583-2149ad197de4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:23.861740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 74/133] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/role_assignments?user.id=a087e0fc3b104c7bb645f80189e77c24&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=c960879a4cc94eabb282279b6c35d693 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:23.862531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:23.940720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 60/134] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:23 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/59184329dff84fd99b155057248ab763/roles/a2ae414a10a54833a1002750699b5d83 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:23.942590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066008-36f8-4b69-8100-ea90cdabad2a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.943122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066008-36f8-4b69-8100-ea90cdabad2a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.943608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066008-36f8-4b69-8100-ea90cdabad2a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.945316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 61/135] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:23.954377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:23.956424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:23.958565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:23.999809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:24.001536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:24.041396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60c0c-f224-428b-8bbe-961758dd72dd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.049605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:24.082112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:24.083665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.084183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.084658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.086324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:24.088539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:24.090908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 75/136] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:23 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:24.096445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.154466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 62/137] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:24.175275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:24.196019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.226542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037cfbd-80fe-4436-a1b9-547d2e415f54 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.261929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:24.262842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.263124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.263395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] PATH_INFO: `/v3/projects/484d9ff52f224bb28e22bba11d369914` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.265149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] RBAC: Authorizing `identity:get_project(project_id=484d9ff52f224bb28e22bba11d369914)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:24.266825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:24.268885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 76/138] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:24 2022] GET /identity/v3/projects/484d9ff52f224bb28e22bba11d369914 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:24.276450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.414391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7b378-bf84-43e1-8545-d1dc62cfff82 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.455799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07f09d-db96-4809-b490-f7313c21bdcb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.478153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:24.491601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:24.495328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.495328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.495328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] PATH_INFO: `/v3/projects/484d9ff52f224bb28e22bba11d369914/users/59184329dff84fd99b155057248ab763/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.495555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:24.495796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] RBAC: Authorizing `identity:create_grant(project_id=484d9ff52f224bb28e22bba11d369914, user_id=59184329dff84fd99b155057248ab763, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:24.497168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b2ed5-34eb-41e1-9124-050ea03abe04 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:24.503095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.503398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.503676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.518480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f02d75a3-f7f4-4005-b722-cac8ce9e0d87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:24.526798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:24.527702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:24.528510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:24.530761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 63/139] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:24.532319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9032-fdf7-4e55-83b8-335e7980cde9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.532814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9032-fdf7-4e55-83b8-335e7980cde9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.533251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e9032-fdf7-4e55-83b8-335e7980cde9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.534715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 64/140] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:24.539582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e9032-fdf7-4e55-83b8-335e7980cde9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.571074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:24.589369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 77/141] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:24 2022] PUT /identity/v3/projects/484d9ff52f224bb28e22bba11d369914/users/59184329dff84fd99b155057248ab763/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:24.593177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.593177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.593177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.625812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:24.627161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:24.724473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e9032-fdf7-4e55-83b8-335e7980cde9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.734696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 78/142] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:24.780660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:24.781467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:24.781764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:24.782048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:24.782848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:24.783165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:24.787788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:24.800338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 65/143] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:24.804220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.815602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:24.961335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:24.966180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897ca78-2637-4925-8dcb-1d5fb9f3856e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.977720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b459f-ec78-4f40-8cd7-08abeb3eb21c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:24.978634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:25.031676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.032561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.032843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.033120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.034716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.040832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.040938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.040938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.040938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.041158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.042396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.047999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.049022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 66/144] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:24 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:25.049938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 79/145] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:25.051945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:25.055851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:25.246868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0c92-35e7-4848-a89d-e5cd1171fd8a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:25.251866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e33b5-856d-438e-b600-e5631449b8db None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:25.332617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.333976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.335016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.335820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.336331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.337265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.338132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.339842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.340346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.347024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.392242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:25.450141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.455802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.486609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 80/146] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:25 2022] POST /identity/v3/users => generated 283 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:25.580174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 67/147] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:25.585816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:25.641255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:25.735427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fcab3-de84-4578-9136-46c25b187d0f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:25.778015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.778959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.779244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.779520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.781037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.782580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16edde-dff8-4651-87e9-15c085b8c14f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.784331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 81/148] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:25 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:25.790336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:25.881006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:25.926239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c62a67-20e8-4f1a-9fc5-fc64e421f7c3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:25.964090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:25.964757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:25.965042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:25.965319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:25.966160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=59184329dff84fd99b155057248ab763) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:25.966476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:25.968499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54482cff-cf81-42d5-9434-3c1f59f987b2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:25.975243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 68/149] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:25 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=66ef983ebbae442b8cab404599a2d333 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:26.234871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dda3-4597-489e-b328-5831bcfcc78a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.235073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dda3-4597-489e-b328-5831bcfcc78a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.235362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a9dda3-4597-489e-b328-5831bcfcc78a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.236895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 82/150] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:26.245561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.246128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.246670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.265634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:26.267878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:26.344128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 69/151] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:26.370072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9dda3-4597-489e-b328-5831bcfcc78a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:26.490655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:26.494611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a9dda3-4597-489e-b328-5831bcfcc78a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:26.537485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:26.538304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.538610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.538892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.554120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-843a5cec-ee60-42fe-8610-8aafc8bb7383 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:26.563066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:26.563926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:26.564887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 83/152] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:26.571722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:26.659845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:26.707885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9b10d-4582-418f-a878-59fec58ebcd4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:26.719764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c76a63-6851-4293-b8a5-a559b19dd210 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.720146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c76a63-6851-4293-b8a5-a559b19dd210 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.720747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c76a63-6851-4293-b8a5-a559b19dd210 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.722178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 84/153] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:26.728434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.728803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.729153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.746653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:26.747602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.748081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.749091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.750027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:26.752547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:26.754207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:26.757527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:26.758847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:26.764760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 70/154] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:26.770794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:26.879306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ce8d6-1ade-40cb-994d-d23f5f340226 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:26.894782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:26.918349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 85/155] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:26.926081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:26.927003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:26.927289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:26.927570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] PATH_INFO: `/v3/users/ecb45340b1984b23904f9e0f1752bcc6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:26.928811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] RBAC: Authorizing `identity:get_user(user_id=ecb45340b1984b23904f9e0f1752bcc6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:26.930271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:26.934830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 71/156] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:26 2022] GET /identity/v3/users/ecb45340b1984b23904f9e0f1752bcc6 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:26.941602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.021339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.071222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:27.079126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13c882-bb85-49fc-b7a7-b921314ec7f0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:27.129859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:27.130546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.130822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.131099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.218814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:27.220330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb590-4858-4bfd-8982-28d34638bb4c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:27.254535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f57e3-df5d-4286-b0c8-4ea3f77bebbd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:27.308700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:27.309901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.310398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.311043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.328750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:27.343391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:27.344335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:27.345111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:27.347722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 72/157] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:27.363447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.378883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:27.392062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 86/158] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:27.393396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cbc46-d90f-4240-82ad-fc7a43fc8c07 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.393680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cbc46-d90f-4240-82ad-fc7a43fc8c07 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.393959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4cbc46-d90f-4240-82ad-fc7a43fc8c07 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.401945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 87/159] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:27.402613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cbc46-d90f-4240-82ad-fc7a43fc8c07 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.432473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:27.495875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde96c5c-d739-42c1-abc6-62177b200884 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:27.543231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:27.543926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.544216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.544500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.545305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:27.546407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:27.547925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c1ffe-ccd5-4e23-887b-403989469408 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:27.548272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4cbc46-d90f-4240-82ad-fc7a43fc8c07 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:27.557581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 73/160] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:27.558642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.558991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.559220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.590615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:27.592665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:27.621790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:27.622704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.623035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.623306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.633952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:27.635483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:27.637402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 88/161] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:27.638180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.745159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 74/162] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:27.746370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.780211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:27.816941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:27.868317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122521-f90d-42ea-a2f7-c50a4dd04123 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:27.926980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:27.928243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:27.928684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:27.929113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] PATH_INFO: `/v3/users/a087e0fc3b104c7bb645f80189e77c24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:27.932709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] RBAC: Authorizing `identity:get_user(user_id=a087e0fc3b104c7bb645f80189e77c24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:27.934376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:27.937069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 89/163] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/users/a087e0fc3b104c7bb645f80189e77c24 => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:27.938016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:27.974547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ed503-a325-4ef3-b425-c03ab1fd1864 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.035800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.036476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.036758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.037028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.037868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=ecb45340b1984b23904f9e0f1752bcc6) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:28.038177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:28.039707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:28.047052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 75/164] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/role_assignments?user.id=ecb45340b1984b23904f9e0f1752bcc6&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:28.060902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.148544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021d5fe-7d2a-4ef1-9602-e9fa1e5c268b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.224664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:28.245265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.246260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.246570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.246855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.252170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:28.261831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391db7f-a386-4c83-a8aa-85318128116c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.272149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ee477e91-8f03-4c2e-8bd1-4e00c7beadb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:28.288410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:28.289359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:28.290169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 90/165] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:28.294772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8634-b257-4662-a2e5-43f09f4d36d9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.295065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8634-b257-4662-a2e5-43f09f4d36d9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.295892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8634-b257-4662-a2e5-43f09f4d36d9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.297361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 91/166] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:28.302066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8634-b257-4662-a2e5-43f09f4d36d9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.332116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.333708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.334189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.334717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.339801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:28.341941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfc7cc-8b1d-449a-86c9-c7527bc56f3f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:28.344643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 76/167] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:28 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:28.346734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.347274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.347755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.393298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:28.394504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:28.500259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8634-b257-4662-a2e5-43f09f4d36d9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.505650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 77/168] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:28.506834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.564629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.565307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.565593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.565868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.566736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:28.566957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:28.568347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:28.581476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 92/169] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:28.582335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.613962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:28.618121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652911b4-0f85-47ac-b0e7-f4cc140c83cc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.654686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.656535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.657011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.657499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/a087e0fc3b104c7bb645f80189e77c24/roles/c960879a4cc94eabb282279b6c35d693` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.661854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, user_id=a087e0fc3b104c7bb645f80189e77c24, role_id=c960879a4cc94eabb282279b6c35d693)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:28.663610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:28.699841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:28.735381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 78/170] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:28 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/users/a087e0fc3b104c7bb645f80189e77c24/roles/c960879a4cc94eabb282279b6c35d693 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:28.736619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.742062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0f36-5b49-409d-9a18-0ddb443dd181 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:28.781995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:28.782796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:28.783171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:28.783362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:28.801706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:28.810283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:28.811829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:28.812644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:28.822546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 93/171] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:28.822546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:28.930990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee16e7-efad-4c5c-a825-3ada1e63ae4c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.000233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.001495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.001969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.002455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.003931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.005749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.007893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbfbcf-046d-4d6e-bb8b-233a38171c92 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.008120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 79/172] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:28 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:29.018470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.024724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.067722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.079436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.080126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.080406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.081645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.082439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:29.082771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.084305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.094175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 94/173] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:29.103937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.189650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec881cc1-ba26-45df-bf54-7c161a11b074 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.252800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.253461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.253738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.254006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.282023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50e586-30a1-4bb8-8af2-6fce98c3d8c9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.337802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.338634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.338948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.339210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] PATH_INFO: `/v3/users/59184329dff84fd99b155057248ab763` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.343371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=59184329dff84fd99b155057248ab763)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.344850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.346980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 95/174] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/users/59184329dff84fd99b155057248ab763 => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:29.353248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.417729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.453850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.455370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124e89fd-e9c7-4543-9632-eba6cc91fe50 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.501519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.532392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b23a4a-93c5-42aa-b804-7a663a557bf5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.579028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.579798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.579963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.580231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.643830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 80/175] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:29.650097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18fa6a-245d-46ae-af8e-04cc6325fb9b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.650707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18fa6a-245d-46ae-af8e-04cc6325fb9b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.651196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18fa6a-245d-46ae-af8e-04cc6325fb9b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.652852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 81/176] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:29.654317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413257-b6a5-42d0-9c92-10d540425ac4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.655428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413257-b6a5-42d0-9c92-10d540425ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.655925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a413257-b6a5-42d0-9c92-10d540425ac4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.657458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 82/177] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:29.658365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413257-b6a5-42d0-9c92-10d540425ac4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.668314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.669807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d3615-8906-4abe-8a73-72a593b4a7e4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.711308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.770499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a413257-b6a5-42d0-9c92-10d540425ac4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:29.777552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 96/178] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:23:29.779019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d50672-d29b-4d49-9a91-1ef0c90a0679 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.779312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d50672-d29b-4d49-9a91-1ef0c90a0679 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.780142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d50672-d29b-4d49-9a91-1ef0c90a0679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.797116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-73d50672-d29b-4d49-9a91-1ef0c90a0679 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:29.797723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:29.798903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-73d50672-d29b-4d49-9a91-1ef0c90a0679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:29.808912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:29.809773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.810054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.810327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.811651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:29.813098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906811b-40fa-46b6-bb9b-42522074da78 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:29.814829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 83/179] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:29.815978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.816260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.817770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.836086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:29.837816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:29.884159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 97/180] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:29.890756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6a3d5-1fbe-4ee9-a29f-7f61fb6f11e6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:29.890756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6a3d5-1fbe-4ee9-a29f-7f61fb6f11e6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:29.890756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6a3d5-1fbe-4ee9-a29f-7f61fb6f11e6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:29.890756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 98/181] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:29.890756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6a3d5-1fbe-4ee9-a29f-7f61fb6f11e6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.915304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 84/182] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:29.916690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:29.963785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.003815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6a3d5-1fbe-4ee9-a29f-7f61fb6f11e6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.034770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf041ad-29c5-4e25-bf53-888ce6a8f971 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.050607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.050607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.050607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.050607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] PATH_INFO: `/v3/projects/484d9ff52f224bb28e22bba11d369914` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.051179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] RBAC: Authorizing `identity:get_project(project_id=484d9ff52f224bb28e22bba11d369914)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.052487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b3d703-1d7e-4de1-b67a-9a9d04f2edbe None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.054183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 99/183] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/projects/484d9ff52f224bb28e22bba11d369914 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.055577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.055860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.056132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.066330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.073515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:30.074728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:30.080970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.081470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.081748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.082025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.082892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=59184329dff84fd99b155057248ab763) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:30.083207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.084657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.090536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 85/184] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:29 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=a2ae414a10a54833a1002750699b5d83 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.091389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.181557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 100/185] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:30.182802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.220312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fefc20-f7ad-4bbd-892f-ceb05e665fe6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.260927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.261724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.262007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.262445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.262585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.278637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:30.288065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:30.289161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 86/186] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:30.290214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.319665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d68dba-f021-44d7-9dbb-1192b6338c7b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.349899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.368024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.369585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.369585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.369585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.370876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.372187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.374306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 101/187] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.377449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.414633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe9b0d-7e1b-42c7-94a3-000b02b24377 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.453822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.455186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.455186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.455186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.455796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=484d9ff52f224bb28e22bba11d369914, user.id=59184329dff84fd99b155057248ab763) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:30.456106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.457563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.474555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 87/188] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/role_assignments?user.id=59184329dff84fd99b155057248ab763&scope.project.id=484d9ff52f224bb28e22bba11d369914&role.id=66ef983ebbae442b8cab404599a2d333 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.474555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.500606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3001d-31ed-4327-ad34-850263b78cdb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.553166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.553998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.554277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.554567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.557154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.569074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:30.578358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:30.579917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:30.581192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:30.585946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 102/189] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:30.586999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.612504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617cc04-8fc4-4fbf-8716-f61de657a33e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.628048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.668978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.669715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.669964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.670285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.671080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:30.671818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.673320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.682931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 88/190] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.683826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.714843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03725b77-ed6c-4f9b-9e7a-527232314be5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.759371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.760046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.760323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.760597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.761304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.762769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.800526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb063b-cf13-4ada-a43f-32d44a1ef863 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.833131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:30.833800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:30.834081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:30.834355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:30.835138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:30.835449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:30.836824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:30.846325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 89/191] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:30.847168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.866547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 103/192] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:30 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:30.866547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:30.893429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.925587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:30.993542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2e693c-2d72-42fc-8fca-b0c36d0fc9d3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:30.999445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f0c14-37ed-4879-b8e8-47e328ae3272 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.040598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.041844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.042126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.042401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.043427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.044505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.044801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.046464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.063455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.064336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.064974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.066262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.066819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 90/193] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:31.068395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 104/194] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:30 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:31.076038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.077884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.194905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.205927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000c17-6916-4437-99d0-6a9e7856cb54 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.208550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebabb4-77f9-4331-a32c-9db29ff01c3a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.219940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.256612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.257513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.258405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.258890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.259231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.259494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.259691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.260064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.350985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.352621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cabd-a19f-4143-9e88-226795930e2b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.361222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.362967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245332ee-3b8a-434d-9923-149baaae90bf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.474210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.490445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 91/195] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:31.508188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b265d-84cc-41f1-bd43-521d9d7a184b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.508188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b265d-84cc-41f1-bd43-521d9d7a184b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.508188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b265d-84cc-41f1-bd43-521d9d7a184b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.508188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 92/196] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:31.508188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b265d-84cc-41f1-bd43-521d9d7a184b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.508551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 105/197] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:31.508551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.508551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.508551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.525122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:31.526335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:31.528370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.598800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 106/198] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:31.599491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.632902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b265d-84cc-41f1-bd43-521d9d7a184b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.678642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.679771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.680301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.680608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.682723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.685518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.687822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 93/199] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:31.688459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.721544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b70ae46-a362-41e8-89b0-34cf82f0c3ce None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.759561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.760794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.762248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.762665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.762943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.764936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.767040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.768757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 107/200] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:31.769678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.781010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:31.808739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8faec-3fae-4e36-aaaf-c4f4a9ba9779 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.846235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.847433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.847905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.848340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.860831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:31.869565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:31.870449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:31.871260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:31.874389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 94/201] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:31.875671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.880145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d5a48-e6d8-4106-8ddc-75d0b2512acf None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:31.915690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:31.916344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:31.916634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:31.917572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:31.917742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=f99c855a7c924db6a06e75ffa6749d30) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:31.918031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:31.919499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:31.925696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 108/202] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/role_assignments?user.id=f99c855a7c924db6a06e75ffa6749d30&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=a2ae414a10a54833a1002750699b5d83 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:31.926008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:31.994327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb1bd4-2dec-4921-8ecc-372806e3ef05 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:32.015360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:32.034462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:32.035632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:32.036114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:32.036653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:32.037855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=f99c855a7c924db6a06e75ffa6749d30) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:32.038380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:32.040127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:32.047581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 95/203] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/role_assignments?user.id=f99c855a7c924db6a06e75ffa6749d30&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:32.064911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:32.101636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70262045-7ad4-42ef-855c-6b3e9ecccac5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:32.142931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:32.143632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:32.143910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:32.144183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:32.144957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:32.145277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:32.146752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:32.157794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 109/204] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:32.162960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:32.284723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb5267-7c85-4fef-a447-190edf964564 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:32.330711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:32.331608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:32.331889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:32.332162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] PATH_INFO: `/v3/users/a087e0fc3b104c7bb645f80189e77c24` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:32.339245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] RBAC: Authorizing `identity:get_user(user_id=a087e0fc3b104c7bb645f80189e77c24)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:32.340703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:32.342395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 96/205] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:32 2022] GET /identity/v3/users/a087e0fc3b104c7bb645f80189e77c24 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:32.347815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:32.479361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc9e1-0b97-4d35-a50f-cd0481b3dfb0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:32.510428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:32.514085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:32.514865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:32.515149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:32.515422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:32.586929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:32.588400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:32.693303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 110/206] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:32.699112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:32.764310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:32.816000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5beb68-ce1e-4fba-ae0b-a5a28c015d42 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:32.851713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:32.852506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:32.852846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:32.853129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:32.854328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:32.855820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903d286-1121-4b81-a050-ede7ffa37572 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:32.857722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 97/207] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:32 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:32.863501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:32.915558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:33.017965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46499671-20b5-4f72-bfb3-b17046ab54d5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:33.064013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:33.064692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.065006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.065285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.066129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270, user.id=a087e0fc3b104c7bb645f80189e77c24) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:33.066443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:33.067921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a10709-18ad-4ad1-ae11-5959bc398056 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:33.079735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 111/208] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:32 2022] GET /identity/v3/role_assignments?user.id=a087e0fc3b104c7bb645f80189e77c24&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=c960879a4cc94eabb282279b6c35d693 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:33.434659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb924-e35e-464d-872b-3d9467dc54e0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.434969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb924-e35e-464d-872b-3d9467dc54e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.435452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eb924-e35e-464d-872b-3d9467dc54e0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.436843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 98/209] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:33.443413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620feb77-db61-4945-8c32-1ecb208d6378 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.443705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620feb77-db61-4945-8c32-1ecb208d6378 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.443986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620feb77-db61-4945-8c32-1ecb208d6378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.475509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-620feb77-db61-4945-8c32-1ecb208d6378 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:33.476817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-620feb77-db61-4945-8c32-1ecb208d6378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:33.588378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 112/210] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:33.667817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eb924-e35e-464d-872b-3d9467dc54e0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:33.667992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6f88b-d010-4302-a562-70465ced1a66 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.667992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6f88b-d010-4302-a562-70465ced1a66 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.671615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6f88b-d010-4302-a562-70465ced1a66 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.676955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 113/211] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:33.680445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.684300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.684814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.718441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:33.721540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:33.756794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:33.817324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 114/212] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:33.825623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eb924-e35e-464d-872b-3d9467dc54e0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:33.841805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:33.844413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:33.863149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:33.864325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.864432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.864688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.877250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:33.885450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:33.886342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:33.887153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 99/213] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:33.892580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:33.947780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf472bcf-843a-4a4b-9d83-e70f478c7267 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:33.981415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:33.983093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:33.983575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:33.984060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:33.985432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:33.987123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:33.993434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 115/214] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:33.994497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.004464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7f0bd-8f13-4d2c-a23a-747d0c50876f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.022233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.044055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.045682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.045985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.046270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.047026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:34.047336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.048850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:34.060495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 100/215] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:34.066590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.113022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e12c-fff8-4f02-8553-6d137a01f328 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.134894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.156468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.157647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.157935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.158219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.159082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.160876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:34.195241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d5702-11f0-4b23-99fe-d2adaa0465d8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.232232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.233551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.234617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.235254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] PATH_INFO: `/v3/users/6cc4958fee1e4435a802cf07abe92eed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.251746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] RBAC: Authorizing `identity:get_user(user_id=6cc4958fee1e4435a802cf07abe92eed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.253619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1389c-f587-4be8-8dc6-18321eb95469 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:34.254656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 116/216] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:33 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:34.262294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 101/217] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:34 2022] GET /identity/v3/users/6cc4958fee1e4435a802cf07abe92eed => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:34.269060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.390927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.396407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3bfb3-8882-4bda-a337-1f376da30774 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.433582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.434326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.434640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.434918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.508283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.509693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:34.586131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111e4b4-12e3-400a-8802-7ad4eeec9a08 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.586451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111e4b4-12e3-400a-8802-7ad4eeec9a08 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.586731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111e4b4-12e3-400a-8802-7ad4eeec9a08 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.588463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 102/218] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:34.594958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.595251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.595531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.598445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.616571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:34.617764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:34.621222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 117/219] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:34.629327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.706280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 103/220] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:34.742948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.769630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40539-e4c1-40a1-b850-217d806adc57 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.781641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.817253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.818861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.819699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.820425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.822506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.824835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c21325-69ec-4ba4-b310-78094df763c2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:34.828038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 118/221] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:34 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:34.829094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73638346-a36a-43fc-bdcf-aab590b05c4b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.829374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73638346-a36a-43fc-bdcf-aab590b05c4b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.829645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73638346-a36a-43fc-bdcf-aab590b05c4b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.830946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 119/222] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:34.835992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73638346-a36a-43fc-bdcf-aab590b05c4b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:34.874457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65da2c-b680-4119-8901-9741ffe52425 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.890849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:34.914373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.915601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.916085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.916556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.930355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f2cb17c2-dfe8-450a-b3cd-5f81c98ed0a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:34.941232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:34.942148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:34.942991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:34.946083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 104/223] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:34.948142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.948580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.949005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.951658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73638346-a36a-43fc-bdcf-aab590b05c4b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:34.973091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:34.975043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:34.993621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:34.994811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:34.995232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:34.995889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:34.997262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=6cc4958fee1e4435a802cf07abe92eed) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:34.997723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:34.999973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.008946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 120/224] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:34 2022] GET /identity/v3/role_assignments?user.id=6cc4958fee1e4435a802cf07abe92eed&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=66ef983ebbae442b8cab404599a2d333 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:35.009951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.117021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 105/225] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:35.134013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa29be8-b962-4c1f-99aa-4165b34950c8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.148276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:35.152386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.177421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.178355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.180438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.180878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.182169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:35.184175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.185799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.197711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:35.206717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 121/226] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:35.212484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.310438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904b8487-6471-4602-9be0-59d50a3ee255 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.366767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13755d-4790-4667-b64a-dd5ad8ba9e99 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.405366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.406255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.406564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.406838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.408140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.409599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.411277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 122/227] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:35.416198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.416849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.417144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.417420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.418924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.440051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:35.448289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:35.449154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:35.449938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 106/228] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:35.465478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.480347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:35.543648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f958c81-7969-4b85-a5b3-18cab4550fc4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.547650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:35.569912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e61af-cdfd-4138-8b61-53266ee79d28 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.582760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.584141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.584635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.585845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.587263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.589159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.591127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 123/229] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:35.602538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.627623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.628289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.628627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.628907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.629666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:35.630028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.631474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.642163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 107/230] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:35.647856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.760352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d853f2c-944f-418f-85c7-f2c98a603397 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.772047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:35.796713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.797621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.797901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.798178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/f99c855a7c924db6a06e75ffa6749d30/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.800350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, user_id=f99c855a7c924db6a06e75ffa6749d30, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.801817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.827961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b16f-d70d-4a0a-9d0b-83339176edd4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:35.867368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:35.868346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:35.868630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:35.868950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:35.870325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:35.871909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:35.874678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 108/231] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:35.882269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 124/232] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:35 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/f99c855a7c924db6a06e75ffa6749d30/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:35.883297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:35.911454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:36.002127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc9174b-079c-48a4-bee8-ebf116dd76fd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:36.041224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:36.042194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.042498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.042886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.044196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:36.046377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6460f6-2234-49c9-ac92-f286a56616af None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:36.048167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 109/233] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:35 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:36.054121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:36.179403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de5d7a-9078-41af-bef6-fefd7ab7f622 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:36.213751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:36.214664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.214940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.215207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/f99c855a7c924db6a06e75ffa6749d30/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.218191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, user_id=f99c855a7c924db6a06e75ffa6749d30, role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:36.219751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b192b8-6959-43b4-a104-1ed730193348 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:36.378054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 125/234] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:36 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/f99c855a7c924db6a06e75ffa6749d30/roles/a2ae414a10a54833a1002750699b5d83 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:36.812990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72c3f-fa4d-41f5-a03a-481f6d38387e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.813315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72c3f-fa4d-41f5-a03a-481f6d38387e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.813615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72c3f-fa4d-41f5-a03a-481f6d38387e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.815281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 110/235] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:36.820264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.820706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.820878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.850371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe1378-d482-45fd-a7a3-bdfa28ee1a3e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.850371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe1378-d482-45fd-a7a3-bdfa28ee1a3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.850371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe1378-d482-45fd-a7a3-bdfa28ee1a3e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.850371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 111/236] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:36.855916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:36.856446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:36.856948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:36.876051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:36.878042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:36.887515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:36.889684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:36.932979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:36.957620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 112/237] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:36.986837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:36.996928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:37.011914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 126/238] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:37.037707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.166003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069123-1f01-460e-b8aa-4a592b11a5a7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.195842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa148b-26d7-48df-827a-d0b7d317f4fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.239612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.240415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.240626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.240737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.240957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.241512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.241801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.242075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.243279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:37.244734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:37.246818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 127/239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:37.252219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.269834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:37.278689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:37.279639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:37.280440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 113/240] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:37.287009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.316208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:37.365254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:37.428255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eaecdc-0482-4b71-a163-fd025996cc77 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.452917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7295e-12bd-4515-8f12-0b81d947060b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.474436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.475161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.475447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.475724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.476497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:37.476809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:37.478472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:37.494434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.495161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.495447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.495725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.496450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:37.498428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cee590-d497-4b76-98e4-df7f817fc912 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:37.503919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 114/241] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:37.512019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.596220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 128/242] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:37 2022] POST /identity/v3/users => generated 293 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:37.630648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac290fa1-f935-4725-9654-65766a460113 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.630995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac290fa1-f935-4725-9654-65766a460113 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.631268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac290fa1-f935-4725-9654-65766a460113 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.633040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 129/243] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:37.640095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.640693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.641188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.645077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5641e80-da47-48d0-9207-ce0d52ed6f68 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.687924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:37.689065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:37.692248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:37.717705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.718639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.718924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] PATH_INFO: `/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:37.749584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] RBAC: Authorizing `identity:get_user(user_id=c7516a0f20104e8b9bf7b7ce9cbb3225)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:37.751699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:37.754262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 115/244] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:37 2022] GET /identity/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:37.759586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.800782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 130/245] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:37.826731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:37.843626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:37.915590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b6c46-c6da-412e-af5d-139613f195ff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.965430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf17b-8e1c-48ac-a658-d9c75b20cdb7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:37.966815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:37.967935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:37.968435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:37.969741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.017913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.019032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.020312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.020767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.037061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:38.045653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:38.050005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:38.051213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:38.052001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:38.055335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 131/246] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:38.062805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.071637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.073357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.159385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.188616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47baf8ee-0136-489e-95ff-061ac7c8d32a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:38.213617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 116/247] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:38.218607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.231728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.232751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.233178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.233714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.234633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:38.235196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.236883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.246824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 132/248] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:38.253987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.336103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.359867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c2cd8-b9ae-476c-922c-8e6616e219c5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:38.399282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6aa1a-5c28-4b79-8466-f9a3343798a0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:38.426562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.427759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.428211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.428650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.431513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.433158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.441179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 117/249] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:38 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:38.446717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.471973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.473144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.473835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.474932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] PATH_INFO: `/v3/users/6cc4958fee1e4435a802cf07abe92eed` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.476640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] RBAC: Authorizing `identity:get_user(user_id=6cc4958fee1e4435a802cf07abe92eed)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.478105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.480146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 133/250] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:38 2022] GET /identity/v3/users/6cc4958fee1e4435a802cf07abe92eed => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:38.485522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.490200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.647738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fee6e3-aa01-4c66-a988-073d728bd28c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:38.681095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7e353-d905-43e4-831f-718dc294fc13 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:38.704117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.705390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.705903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.706388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.711305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.713028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.715034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 134/251] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:38 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:38.728565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:38.735819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:38.736502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.736787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.737060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.737893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=c7516a0f20104e8b9bf7b7ce9cbb3225) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:38.738206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:38.742821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1f67c-8bcd-4a4c-8f22-a83d1f0d5f26 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:38.759253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 118/252] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:38 2022] GET /identity/v3/role_assignments?user.id=c7516a0f20104e8b9bf7b7ce9cbb3225&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=c960879a4cc94eabb282279b6c35d693 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:38.850871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f18a-1331-4165-b235-f1f7aa02cbc5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.851216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f18a-1331-4165-b235-f1f7aa02cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.851430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07f18a-1331-4165-b235-f1f7aa02cbc5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.853068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 119/253] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:38.868037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:38.868354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:38.868593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:38.896408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.911010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:38.912238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:38.945437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:38.958556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a52065-a6cf-45d2-9e62-3d5b5b531d72 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:39.040720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:39.041641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.041922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.042198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/6cc4958fee1e4435a802cf07abe92eed/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.048312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, user_id=6cc4958fee1e4435a802cf07abe92eed, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:39.049982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ee363-378b-4e06-9390-b486a42a1a9e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:39.082309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 120/254] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:39.106321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.109317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 135/255] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:38 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/6cc4958fee1e4435a802cf07abe92eed/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:23:39.150671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4208127-f8c9-444f-8d34-e67810ac78d2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.151013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4208127-f8c9-444f-8d34-e67810ac78d2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.151237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4208127-f8c9-444f-8d34-e67810ac78d2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.153070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 136/256] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:39.161149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.161437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.161871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.207637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:39.208861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:39.313774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a107b1-444a-4d84-a78d-2a513f170433 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:39.360598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 137/257] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:39.387148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:39.387995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.388368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.388643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.396865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.401819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:39.409871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:39.410809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:39.411598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 121/258] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:39.417049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.430577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:39.528068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:39.587621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a517b-9a34-4261-96f0-bc2b537825c7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:39.595734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436587-60f1-4e41-9f85-2a0e554bf7b9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:39.631852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:39.632545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.632828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.633105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.633866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:39.634174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:39.635621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:39.651037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 122/259] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:39.657414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.661181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:39.662012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.662294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.662600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.690148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:39.699051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:39.699959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:39.701026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 138/260] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:39.726120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.747022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:39.800727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec214f-8013-4b37-b34f-7b433f02b7c1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:39.942820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:39.943618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:39.943896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:39.944168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:39.945515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:39.947025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:39.948671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 123/261] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:39.955001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:39.999486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c1b15-8d91-448c-8e99-946415c2d828 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:40.062896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:40.107804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:40.108495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.108774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.109045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.109920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:40.110228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:40.111649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:40.130099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 139/262] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:40.136971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:40.235757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffa3aad-7080-40dc-a967-c1d25464b582 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:40.307302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:40.307964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.308244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.308518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.330875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:40.394663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042b01a-0015-4275-ac56-a48e74ded0fd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:40.476155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:40.477038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.477314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.477587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] PATH_INFO: `/v3/users/f99c855a7c924db6a06e75ffa6749d30` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.479001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] RBAC: Authorizing `identity:get_user(user_id=f99c855a7c924db6a06e75ffa6749d30)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:40.480459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:40.482982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 140/263] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:40 2022] GET /identity/v3/users/f99c855a7c924db6a06e75ffa6749d30 => generated 283 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:40.487480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:40.488918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:40.504947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:40.601734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:40.683487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 124/264] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:40.692624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:40.711605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb92047-5e3b-4ba2-a514-411a419672ed None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:40.782503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:40.797228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:40.797894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.798174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.798447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.891907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372fb8-8e41-4f18-9d01-a007320ac193 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:40.924819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:40.927022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8eb749-1c10-4e63-bda9-f25084a6fd55 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:40.934350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:40.936839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.937182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.937527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.938960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:40.940560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dcd31b-855d-4dd6-a07f-8cee8e3b52c2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:40.942855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 125/265] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:40 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:40.944318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db67c-bf62-457f-a64b-dded71b10aee None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.944672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db67c-bf62-457f-a64b-dded71b10aee None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.945017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6db67c-bf62-457f-a64b-dded71b10aee None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.946782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 126/266] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:40.947960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783e59-e200-46ff-82b1-c0996241612e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:40.948487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783e59-e200-46ff-82b1-c0996241612e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:40.948952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec783e59-e200-46ff-82b1-c0996241612e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:40.952184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 127/267] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:40.956889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec783e59-e200-46ff-82b1-c0996241612e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:40.987666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.047328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 141/268] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:23:41.050377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.051074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.051569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.073498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:41.075037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:41.083917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec783e59-e200-46ff-82b1-c0996241612e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.094299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.130650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.132044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.132870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.133374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.134595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=f99c855a7c924db6a06e75ffa6749d30) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:41.135085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.137222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce72788-6ca9-436e-be3c-d3b010556d69 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.145916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 128/269] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:40 2022] GET /identity/v3/role_assignments?user.id=f99c855a7c924db6a06e75ffa6749d30&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.148178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.148885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.149327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.152438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 142/270] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:41.153466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.171706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:41.174080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:41.259440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.273659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 129/271] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:41.275141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.312882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a2e6c-c9a7-4734-af25-1323ae8adc16 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.378702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.379873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.380445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.380913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.382592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.385214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.387914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 143/272] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.389671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.448413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.453269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9e75-f12e-4075-a8e2-9bd8a417565f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.495980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.496783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.497062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.497371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.508660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:41.515243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86234379-b972-4a52-81b6-cb0bb74648b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:41.517006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:41.517855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:41.518678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:41.519275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 130/273] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:41.519978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.562015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.563436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.564218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.564847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.566332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.568093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.571272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 144/274] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.572293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.598237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.653218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa02175-fdaa-41d9-bc13-476c67a455bf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.689584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.690240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.690552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.690827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.692124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=f99c855a7c924db6a06e75ffa6749d30) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:41.692455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.693908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.699724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 131/275] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/role_assignments?user.id=f99c855a7c924db6a06e75ffa6749d30&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=a2ae414a10a54833a1002750699b5d83 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.699724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.710823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb519-1d70-4aac-b3ab-7e6cf732652e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.735779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.758783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.759579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.759943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.760287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.761713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:41.762100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.765119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b45f4-614d-45f3-b71f-26e45968c95b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.777791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 145/276] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.780264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d56445-d166-4a90-b329-5e57999ee46c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.780623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d56445-d166-4a90-b329-5e57999ee46c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.781127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d56445-d166-4a90-b329-5e57999ee46c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.784094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 146/277] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:41.784973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d56445-d166-4a90-b329-5e57999ee46c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:41.822327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f424bd-dfd3-4c3a-ac46-0295172f2c6b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.862494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.863602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.864123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.864495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.865360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.867218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7ddcb-d2ab-4879-8eef-9dfbb62f75e8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.892749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:41.912642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d56445-d166-4a90-b329-5e57999ee46c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:41.945323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 132/278] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:23:41 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:41.946814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.947336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.947410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.950012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:41.951522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.951802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.952073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] PATH_INFO: `/v3/users/465a45b02ffa4538b73a49e7d4fa81a1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.965892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:41.967226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:41.968965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] RBAC: Authorizing `identity:get_user(user_id=465a45b02ffa4538b73a49e7d4fa81a1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:41.970457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1396c-13da-42d3-9dd4-2fd677b37694 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:41.972435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 147/279] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:41 2022] GET /identity/v3/users/465a45b02ffa4538b73a49e7d4fa81a1 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:41.979884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f386-ada3-41b9-8631-c888955eb75d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.979884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f386-ada3-41b9-8631-c888955eb75d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.979884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f386-ada3-41b9-8631-c888955eb75d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:41.979884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 148/280] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:41.981552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:41.981863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:41.982140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.001010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:42.002265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:42.045744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 133/281] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:42.052168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.069570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.086626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 149/282] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:42.087835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.167827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.183722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4aa027-cd88-4272-a85b-7fecd5e2c1cd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.214810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36335a75-c38f-4c5c-bd57-3a67783459ed None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.228695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.229366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.229695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.229928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.230913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, user.id=465a45b02ffa4538b73a49e7d4fa81a1, scope.system=all) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:42.231229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.232986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.239336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 134/283] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/role_assignments?user.id=465a45b02ffa4538b73a49e7d4fa81a1&scope.system=all&role.id=66ef983ebbae442b8cab404599a2d333 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:42.243694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.261430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.262237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.262544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.263387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.281871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:42.291562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:42.292517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:42.293301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:42.293899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 150/284] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:42.300558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.350541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.394665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe01dd1a-afd5-48bd-9f95-518a3063c686 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.424323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53995de-75df-40e8-ab3f-2d76ffdb9f0b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.439242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.441248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.441667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.442092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.456607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.460824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:42.465396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.466717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.467061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.471879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.471879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:42.471879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.471879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:42.472137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:42.472970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:42.473773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 135/285] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:42.479771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.485278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 151/286] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:42.486720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.599028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1b367-3b15-45d2-b0d6-5ed4d46c4760 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.608178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd3111-f4f6-4080-843c-6fd9e60d0eda None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.635047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.635773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.636060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.636334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.637082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:42.637390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.639467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.640940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.647151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.648032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.648316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.648591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] PATH_INFO: `/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.649796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] RBAC: Authorizing `identity:get_user(user_id=c7516a0f20104e8b9bf7b7ce9cbb3225)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.650230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 136/287] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:42.655426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.657432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 152/288] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:42.658705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.678378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.768598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f7a51-919f-4845-9fc0-146e00815d2b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.798350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.828420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.829310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.829591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.829868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] PATH_INFO: `/v3/users/6cc4958fee1e4435a802cf07abe92eed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.832231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] RBAC: Authorizing `identity:get_user(user_id=6cc4958fee1e4435a802cf07abe92eed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.833698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.835530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 137/289] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/users/6cc4958fee1e4435a802cf07abe92eed => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:42.840857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:42.876984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3782ca-c698-436d-8a8b-5a23d89c0174 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:42.937266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:42.938138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:42.938411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:42.938720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:42.940592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:42.942035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:42.948478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 153/290] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:42.949071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:42.951548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:43.026406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa5c35-83ec-430a-874e-24377d31738d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:43.085715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:43.086607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:43.086998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:43.087371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:43.110770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781d832-a2f8-4d8d-b2c0-5e7b596dd98a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:43.163938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:43.172617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:43.173883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:43.174323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:43.174789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/c7516a0f20104e8b9bf7b7ce9cbb3225/roles/c960879a4cc94eabb282279b6c35d693` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:43.180218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, user_id=c7516a0f20104e8b9bf7b7ce9cbb3225, role_id=c960879a4cc94eabb282279b6c35d693)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:43.180218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:43.203942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:43.205497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:43.261718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 154/291] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:23:42 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/users/c7516a0f20104e8b9bf7b7ce9cbb3225/roles/c960879a4cc94eabb282279b6c35d693 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:43.272299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:43.327691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 138/292] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:43.333005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:43.444918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de561a0-3c08-4814-8ba4-b49df477db74 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:43.483224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:43.484468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:43.484949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:43.485417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:43.486828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:43.488488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4b222-630e-479e-b4c9-334f37b758af None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:43.490789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 155/293] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:43 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:43.496213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:43.631903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566dc53-c716-4b2e-be05-1e24af90474c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:43.684602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:43.685790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:43.686069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:43.686343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:43.687204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=6cc4958fee1e4435a802cf07abe92eed) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:43.687519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:43.688970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e3031-9e96-4173-8ded-7f9926dd0bd9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:43.695256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 139/294] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:43 2022] GET /identity/v3/role_assignments?user.id=6cc4958fee1e4435a802cf07abe92eed&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=66ef983ebbae442b8cab404599a2d333 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:44.562609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3901-dac8-4246-bd64-f7c909cbe507 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.563114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3901-dac8-4246-bd64-f7c909cbe507 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.563189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3901-dac8-4246-bd64-f7c909cbe507 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.564775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 156/295] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:44.570133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.571005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.571488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.589062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:44.590633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:44.596861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99227-da88-493a-ac21-abdf7ddc99a5 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.597403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99227-da88-493a-ac21-abdf7ddc99a5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.597889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99227-da88-493a-ac21-abdf7ddc99a5 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.599518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 157/296] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:44.604764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.605266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.605743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.617603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:44.624030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:44.625952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:44.672747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 140/297] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:44.688876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:44.697343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:44.708106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 158/298] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:44.735197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:44.808856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a45bf-2747-4fa6-9e44-5ffe1019b36f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:44.840244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212610f8-0b5b-4f57-acd0-88e007e04cee None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:44.845297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:44.846271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.846734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.847263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.848431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:44.850024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab2030-33db-40fc-98e8-90593eee72cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:44.852004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 141/299] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:44.853315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f144e2-044e-4589-9e5f-f21dc1ffd5f1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.853708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f144e2-044e-4589-9e5f-f21dc1ffd5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.853987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f144e2-044e-4589-9e5f-f21dc1ffd5f1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.855245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 142/300] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:44.859549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f144e2-044e-4589-9e5f-f21dc1ffd5f1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:44.879549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:44.880270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.880547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.880814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.889993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:44.892495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2f838205-aa2a-4ba9-b49a-2cd37d7a2a7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:44.900363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:44.907417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 159/301] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:44.908389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:44.909129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:44.909129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:44.920239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:44.921410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:44.963241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:44.980033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f144e2-044e-4589-9e5f-f21dc1ffd5f1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:44.991076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 160/302] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:45.002682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.026854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.027665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.027945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.028733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] PATH_INFO: `/v3/groups` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.029450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.030910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.102437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14856ae5-043b-4be9-aa89-e9370b39eb61 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.126951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 143/303] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Jun 24 14:23:44 2022] POST /identity/v3/groups => generated 229 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:23:45.127671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.150611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.151923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.152913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.153392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.154460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:45.155025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.156646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.170956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 161/304] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:45.176744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.219136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.253469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.273365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9adce-e8a6-4505-8f2f-5b2317bc2691 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.299859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a2a3d-8a74-418a-8cf2-d0862ba20c7f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.312528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.313769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.314199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.314677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.330889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:45.339698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:45.340656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:45.342148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 144/305] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:45.348136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.348284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.348377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.348377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.348377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] PATH_INFO: `/v3/users/527f9c1e55df4ffd8740438d37216811` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.366095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] RBAC: Authorizing `identity:get_user(user_id=527f9c1e55df4ffd8740438d37216811)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.367867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.369820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 162/306] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/users/527f9c1e55df4ffd8740438d37216811 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:45.382631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.483886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.486485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5df125-05d9-4584-bfe5-03b842a60fb5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.490737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afcd93-3131-4f4a-9957-7a1119299e91 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.522241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.523214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.523492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.523813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.524561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:45.524863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.525699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.526242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec9a19-2343-46ae-bb19-56064634364a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.526399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.526675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.526954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.527787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, user.id=527f9c1e55df4ffd8740438d37216811, scope.system=all) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:45.528107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.530081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05ba01-a8c0-46d5-9517-f41538ef92b9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.535415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 163/307] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/role_assignments?user.id=527f9c1e55df4ffd8740438d37216811&scope.system=all&role.id=c960879a4cc94eabb282279b6c35d693 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:45.535707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 145/308] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:45.537016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208176ab-9c4b-436d-a9f4-14e4d74d9350 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.537379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208176ab-9c4b-436d-a9f4-14e4d74d9350 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.537729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208176ab-9c4b-436d-a9f4-14e4d74d9350 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.539174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 164/309] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:45.546265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.546596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.547070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.547183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208176ab-9c4b-436d-a9f4-14e4d74d9350 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.570011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:45.571586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:45.575676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.675274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 146/310] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:45.690881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208176ab-9c4b-436d-a9f4-14e4d74d9350 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.699850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.748462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.749297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.749615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.749882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] PATH_INFO: `/v3/users/465a45b02ffa4538b73a49e7d4fa81a1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.751425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] RBAC: Authorizing `identity:get_user(user_id=465a45b02ffa4538b73a49e7d4fa81a1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.754509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:45.756656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 165/311] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/users/465a45b02ffa4538b73a49e7d4fa81a1 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:45.762303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.800120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.826793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768b5fa-2653-4dc9-a3e8-9bf26b3062e4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.865569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.866698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.867145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.867545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.875136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:45.879319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:45.889658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:45.890577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:45.891381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:45.893820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 147/312] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:45.899119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:45.900122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790b98f-ff6b-43ee-9a85-a3db3e07bfb8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:45.939530 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:45.940416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:45.940701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:45.940978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] PATH_INFO: `/v3/system/users/465a45b02ffa4538b73a49e7d4fa81a1/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:45.942474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=465a45b02ffa4538b73a49e7d4fa81a1, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:45.944009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.009587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a0645-c7e5-43ba-b38e-307ade5cf589 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:46.042911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:46.043576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:46.044440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:46.044722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:46.045466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:46.045768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:46.047259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.047465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 166/313] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Jun 24 14:23:45 2022] PUT /identity/v3/system/users/465a45b02ffa4538b73a49e7d4fa81a1/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:46.060154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 148/314] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:46.066394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:46.173368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:46.183853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2417fec4-d4f7-4b88-b3f3-1d7067d46e13 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:46.221349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:46.222230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:46.222551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:46.222858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] PATH_INFO: `/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:46.225248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] RBAC: Authorizing `identity:get_user(user_id=c7516a0f20104e8b9bf7b7ce9cbb3225)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:46.226829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.228601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 167/315] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:46 2022] GET /identity/v3/users/c7516a0f20104e8b9bf7b7ce9cbb3225 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:46.234923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:46.299217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:46.343289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba34fb1-e7e2-4b63-925e-9b13da53efdb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:46.380146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:46.380816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:46.381095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:46.381368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:46.453766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:46.455308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.517152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:46.564272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 149/316] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:46.569904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:46.674162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aea785-65c2-4e06-b029-7fe77d77c044 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:46.716577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:46.717461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:46.717742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:46.718016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:46.719263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:46.720704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31ca7-4f92-4cfc-8a1e-f3200b765981 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.722826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 168/317] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:46 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:46.729009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:46.831929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea635f7-7b01-4867-a641-fe1a921306ec None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:46.871181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:46.871870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:46.872685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:46.872957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:46.873867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250, user.id=c7516a0f20104e8b9bf7b7ce9cbb3225) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:46.874182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:46.875646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f612c501-6d58-4bd7-a9f6-7675176e6624 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:46.881983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 150/318] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:23:46 2022] GET /identity/v3/role_assignments?user.id=c7516a0f20104e8b9bf7b7ce9cbb3225&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=c960879a4cc94eabb282279b6c35d693 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:47.345070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a465-8de8-41f5-a842-1c571d5ccd02 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.345464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a465-8de8-41f5-a842-1c571d5ccd02 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.345739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a465-8de8-41f5-a842-1c571d5ccd02 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.347187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 169/319] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:47.352184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.352477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.352756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.370626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:47.372541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:47.391124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:47.440759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 151/320] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:47.462860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86a465-8de8-41f5-a842-1c571d5ccd02 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:47.491041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:47.566431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d86a465-8de8-41f5-a842-1c571d5ccd02 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:47.601216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:47.601969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.602253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.602550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.603711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:47.605156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ecefd-0c3a-489b-a89c-0bae9940d91a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:47.607141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 170/321] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:47.612445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:47.634192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a75f-8f9b-4ce5-9603-ef8865aafd94 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.634469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a75f-8f9b-4ce5-9603-ef8865aafd94 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.634783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a75f-8f9b-4ce5-9603-ef8865aafd94 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.636353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 171/322] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:47.641612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.641895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.642167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.660008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:47.661173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:47.718965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7b07-a5c8-433b-a9b8-c50d2e283dfe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:47.731158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 172/323] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:47.753965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:47.754718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:47.755269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.755544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.755827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] PATH_INFO: `/v3/groups` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.756737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:47.759080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:47.778607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:47.846840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 152/324] 10.222.0.23 () {60 vars in 1134 bytes} [Fri Jun 24 14:23:47 2022] POST /identity/v3/groups => generated 226 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:23:47.859005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939bf63-a7e2-49a6-8706-b4d95452b2e9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:47.899739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:47.900563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:47.900898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:47.901175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:47.913406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:47.921637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:47.922933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:47.928814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 173/325] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:47.931583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:47.951710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:48.046009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cbe01-3853-4ca1-9901-bcffe4eb5348 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.081054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.081733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.082018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.082297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.083086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:48.083399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:48.084789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f68ac-5f3c-4e29-93fb-22115021cffd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:48.093705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 153/326] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:48.099409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:48.175186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f0d-208f-4e23-9770-665a5058804f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.175484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f0d-208f-4e23-9770-665a5058804f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.175835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f0d-208f-4e23-9770-665a5058804f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.177457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 154/327] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:48.183254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.183889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.185032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.203586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:48.204805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:48.206218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:48.220520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334eed68-e70f-4cf6-a6ec-6d03b328af1d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.264145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.265171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.265462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.265742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] PATH_INFO: `/v3/users/527f9c1e55df4ffd8740438d37216811` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.267047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] RBAC: Authorizing `identity:get_user(user_id=527f9c1e55df4ffd8740438d37216811)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:48.268645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:48.270842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 174/328] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:48 2022] GET /identity/v3/users/527f9c1e55df4ffd8740438d37216811 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:48.278859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:48.288628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 155/329] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:48.314598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:48.323836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:48.413955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc315fc1-f40b-402d-bd6d-436644a753cd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.437930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097eeb8-e444-4ec0-b4d8-50452b9cb9ff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.456030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.457249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.457249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.457563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] PATH_INFO: `/v3/system/users/527f9c1e55df4ffd8740438d37216811/roles/c960879a4cc94eabb282279b6c35d693` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.459109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=527f9c1e55df4ffd8740438d37216811, role_id=c960879a4cc94eabb282279b6c35d693)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:48.460484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:48.483154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.483971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.484258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.484533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.496465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:48.504542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:48.505460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:48.506245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 156/330] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:48.511310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:48.546538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 175/331] 10.222.0.23 () {58 vars in 1334 bytes} [Fri Jun 24 14:23:48 2022] PUT /identity/v3/system/users/527f9c1e55df4ffd8740438d37216811/roles/c960879a4cc94eabb282279b6c35d693 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:48.596359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:48.641142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbeb09a-4258-4462-81eb-067957669f3d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.679876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.680611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.680898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.681177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.681942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:48.682251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:48.683735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:48.697082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 157/332] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:48.702617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:48.734850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:48.909072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ca221-ab0d-4b3c-82a7-9f6551c6c293 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:48.959921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:48.960844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:48.961205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:48.961555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] PATH_INFO: `/v3/users/465a45b02ffa4538b73a49e7d4fa81a1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:48.962901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] RBAC: Authorizing `identity:get_user(user_id=465a45b02ffa4538b73a49e7d4fa81a1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:48.964436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707800d-fb6d-47e7-b01a-7bb1fef78b66 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:48.967192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 176/333] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:48 2022] GET /identity/v3/users/465a45b02ffa4538b73a49e7d4fa81a1 => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:48.972690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:49.101877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b21e65-c412-4170-991f-d907a47c91a8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:49.137888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:49.148734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:49.149392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:49.151146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:49.151146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:49.151146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] RBAC: Adding query filter params (role.id=66ef983ebbae442b8cab404599a2d333, user.id=465a45b02ffa4538b73a49e7d4fa81a1, scope.system=all) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:49.151146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:49.152560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bed4ad-afa4-40b6-8d0f-77c7853cda6d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:49.167352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 158/334] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Jun 24 14:23:48 2022] GET /identity/v3/role_assignments?user.id=465a45b02ffa4538b73a49e7d4fa81a1&scope.system=all&role.id=66ef983ebbae442b8cab404599a2d333 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:50.623275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a83eb-10e1-4f33-97a1-310458c9bef9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:50.624256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a83eb-10e1-4f33-97a1-310458c9bef9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:50.624748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a83eb-10e1-4f33-97a1-310458c9bef9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:50.626692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 177/335] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:50.631720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540b79e-8284-4b31-9902-679112e672cf None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:50.632121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540b79e-8284-4b31-9902-679112e672cf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:50.632290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540b79e-8284-4b31-9902-679112e672cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:50.654925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6540b79e-8284-4b31-9902-679112e672cf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:50.656141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6540b79e-8284-4b31-9902-679112e672cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:50.660515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c66f-e5f2-434d-ae7a-5c3b3077e08b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:50.661033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c66f-e5f2-434d-ae7a-5c3b3077e08b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:50.661514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2c66f-e5f2-434d-ae7a-5c3b3077e08b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:50.663107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 178/336] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:50.668473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:50.669034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:50.669504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:50.705383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:50.706614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:50.768788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 159/337] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:50.800565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540b79e-8284-4b31-9902-679112e672cf None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:50.824395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 179/338] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:50.870820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:50.898959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.013315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540b79e-8284-4b31-9902-679112e672cf None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.018069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3dc90-79bc-4b2b-9566-c09803543d76 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.037857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.066784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.067651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.067933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.068208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.091286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:51.104555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.105374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.105651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.106547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:51.110430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:51.111316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:51.112094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:51.114342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 160/339] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:51.122931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:51.139185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:51.147074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:51.147927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:51.149346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 180/340] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:51.158865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:51.293874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.322613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364273f-47c3-4b46-b5f7-3c39a88e97c8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.324471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ebab7-a255-47ab-825c-96ac8cdfc6be None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.382663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.383483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.383772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.383874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.384065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.384495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.384777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.384875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:51.385052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.385145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:51.385818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:51.386128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:51.386502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:51.391670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:51.401004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 181/341] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:51.407369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:51.410119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 161/342] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:51.419937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:51.477800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.567228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcda81-e896-49f2-b813-722d3df9eb9b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.592138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf8d7b-245d-4de2-9542-9bbfa93d7c2a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:51.618106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.619161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.619637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.620265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.661417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:51.663691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.664429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.666100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.675476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.744851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:51.749724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29349350-be28-429a-97db-1c692d87a9c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:51.796571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:51.842933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:51.844490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785500b9-0553-41bb-b5c5-ad71a164bb8e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:51.868815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 162/343] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:23:51.870589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129bd39-73af-430d-8041-26d3430bfadb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.871078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129bd39-73af-430d-8041-26d3430bfadb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.871794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129bd39-73af-430d-8041-26d3430bfadb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:51.873743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 163/344] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:51.878410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129bd39-73af-430d-8041-26d3430bfadb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:51.979813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 182/345] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:51.985845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:51.986131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:51.986407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.008160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:52.008337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129bd39-73af-430d-8041-26d3430bfadb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.009338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:52.040026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.048138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.049011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.049295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.049620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.051770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.052535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cf29b-07fe-44a1-ba68-29eb4d091c0a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.054995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 164/346] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:51 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.056333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07c3c-1470-4ee0-bf4b-d9fa15fafc41 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.056619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07c3c-1470-4ee0-bf4b-d9fa15fafc41 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.056899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07c3c-1470-4ee0-bf4b-d9fa15fafc41 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.063609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 165/347] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:52.064407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf07c3c-1470-4ee0-bf4b-d9fa15fafc41 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.078192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.086934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 183/348] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:52.087788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.230436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf07c3c-1470-4ee0-bf4b-d9fa15fafc41 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.245967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646e672-4abe-4a09-8af4-fbc2d9921247 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.272841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.273702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.273979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.274248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.275473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.276918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da998ae-a3f6-4268-82aa-3b599f3f03dd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.282688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 166/349] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.283938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592986-702f-4d43-abc8-a69954caa922 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.284226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592986-702f-4d43-abc8-a69954caa922 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.284500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d592986-702f-4d43-abc8-a69954caa922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.289746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.293449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.293449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.293449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.303790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0d592986-702f-4d43-abc8-a69954caa922 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:52.304044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.304979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d592986-702f-4d43-abc8-a69954caa922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:52.305522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.310173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 184/350] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.311316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.349963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.389637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 167/351] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:52.391048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8da9-8172-4556-9364-2a5f19d3687a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.391278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8da9-8172-4556-9364-2a5f19d3687a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.391555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d8da9-8172-4556-9364-2a5f19d3687a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.392974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 168/352] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:52.394259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8da9-8172-4556-9364-2a5f19d3687a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.401792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.474477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69decc-a818-4205-87c2-909a924bef3c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.524032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.524759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.525040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.525312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.527393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d8da9-8172-4556-9364-2a5f19d3687a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.541718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:52.549562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:52.550703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:52.551550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 185/353] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:52.552216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.572364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.573362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.573642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.573914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.576387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.577983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee06edc-45d9-4bd3-aa4f-77669ae0ecc3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.580103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 169/354] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.585765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.586050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.586320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.605407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:52.607126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:52.673346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.687661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef214c-b3b6-4f98-982c-2d76c0c0f0ad None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.694318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 170/355] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:52.695087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.697683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.732760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.733425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.733703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.733978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.736005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:52.736329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.737788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.745662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 186/356] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=a2ae414a10a54833a1002750699b5d83 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.747251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.827391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69116ff4-baf8-475a-a276-b75fee20277b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.870502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d6ac9-bb8b-4b89-9308-8c5cf1d76b5b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:52.871118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.872571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.872860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.873140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.887466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:52.895463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:52.896404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:52.897191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 171/357] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:52.897884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.913555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:52.914813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:52.915098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:52.915374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:52.916509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:52.916822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:52.919072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:52.929517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 187/358] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:52.930206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:52.954096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:52.984868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.037204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f50e9a-ca4f-4a1b-b3e0-c0635c7f1547 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.072803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12002634-36df-43d4-8259-22ea6dac8ba5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.078323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.079218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.079649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.080093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.081083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:53.081548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.083451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fccbdf2-56b2-400e-aba0-85924b70fade None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.091160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 172/359] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=66ef983ebbae442b8cab404599a2d333 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:53.092688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f734-774f-4c29-b8dc-962542c3b152 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.093121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f734-774f-4c29-b8dc-962542c3b152 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.093553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34f734-774f-4c29-b8dc-962542c3b152 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.095080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 173/360] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:53.095960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34f734-774f-4c29-b8dc-962542c3b152 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.125246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.126456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.126892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.127604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.143624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:53.153993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:53.155007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:53.155821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 188/361] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:53.156998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.229798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34f734-774f-4c29-b8dc-962542c3b152 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.261791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.268586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.276783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.278170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.278703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.279059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.279955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:53.280333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.281885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821c6a4-b49e-46af-aa59-f15adf430aa8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.288521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbb9bc-5208-47a2-8966-0e58b60c742c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.291930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 174/362] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:53.298171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.298470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.298780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.316842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:53.318680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:53.328552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.329458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.329751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.330032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.407171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 175/363] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:53.408091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.429065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.430599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.532119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55d56b-c126-4877-998f-637a23b6faec None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.535878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.552947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.569068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 189/364] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:53.570392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.573492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.574244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.575127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.575403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.576159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:53.576462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.577854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.589392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 176/365] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:53.590927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.699877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9cb2-76c3-42a4-a674-d077fc7ab345 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.731634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9730-99be-4914-8b6c-89f90a754229 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.741370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.742304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.742630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.742910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] PATH_INFO: `/v3/users/527f9c1e55df4ffd8740438d37216811` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.744123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] RBAC: Authorizing `identity:get_user(user_id=527f9c1e55df4ffd8740438d37216811)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.745583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.748153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 190/366] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/users/527f9c1e55df4ffd8740438d37216811 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:53.755628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.779914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.780725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.781006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.781281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.797086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:53.805469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:53.806385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:53.807209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:53.810181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 177/367] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:53.810181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.820694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.863220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:53.894477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257814a-3378-41e0-a497-8626ee468fb6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.929917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d7bcb-7050-4529-a511-e81766e9d4c3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:53.938421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.939325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.939607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.939881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:53.941083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:53.943234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:53.945855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 191/368] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:53.947244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:53.976811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:53.977365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:53.977649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:53.977926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.070964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905bc80-1f7e-4dfa-9aa5-095eaed7d6a3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.075046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.078395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.090848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.110603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.111313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.111599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.111929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.112773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] RBAC: Adding query filter params (role.id=c960879a4cc94eabb282279b6c35d693, user.id=527f9c1e55df4ffd8740438d37216811, scope.system=all) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:54.113091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.114623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.128908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.132452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 192/369] 10.222.0.23 () {56 vars in 1308 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/role_assignments?user.id=527f9c1e55df4ffd8740438d37216811&scope.system=all&role.id=c960879a4cc94eabb282279b6c35d693 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.133349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.219175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 178/370] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:54.220184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.315795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf9e99-09fc-4c07-8b7d-b84972d36a41 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.388255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.389202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.389476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.389991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.391172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:54.391485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.392874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.395692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934cf3e9-7c45-40be-8608-4f76716060ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.410653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 193/371] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.411548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.443624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.444522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.445097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.445289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.447219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.448794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.452008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 179/372] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.453186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.453936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.467341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.542337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3643a-46f6-41c1-b4b4-5d25000a8684 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.581206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68edff5-74e2-4c19-8ec1-6af423816364 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.585441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.586311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.586617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.586898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.588094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.589555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.592744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 194/373] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.593637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.627361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.628411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.628700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.629255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.710834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.712302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.718905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.722729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0543a-93a9-4b4f-bfae-eeec78457292 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.723500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.764629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.765289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.765570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.765845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.766701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:54.767010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.768440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.774964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 195/374] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=66ef983ebbae442b8cab404599a2d333 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.775916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.834661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 180/375] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:23:54.840595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:54.913627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f67f6-6689-4f1b-8cc8-d24ece52fc72 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.961245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ba901-c876-4e57-b067-4e9b5b53b004 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:54.962224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:54.964550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:54.965680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:54.966349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:54.967830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:54.969493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:54.972204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 196/376] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:54.978146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:54.979129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.026135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.027202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.027519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.027797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] PATH_INFO: `/v3/projects/7486fcd7991c440786c9d72536ea0aed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.029617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=7486fcd7991c440786c9d72536ea0aed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:55.031152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:55.037831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 181/377] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/projects/7486fcd7991c440786c9d72536ea0aed => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:55.039178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.040189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:55.144384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a0dfa-a23d-4b99-b2ef-abbf8422e3d3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:55.180103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.181257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.181685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.182102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] PATH_INFO: `/v3/groups/fce1f1021a434d1b8a779932c19fcc75` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.182412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e649e73-cfc5-4e51-97c6-d11faf069057 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:55.189837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] RBAC: Authorizing `identity:get_group(group_id=fce1f1021a434d1b8a779932c19fcc75)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:55.191558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:55.198313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 182/378] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:55 2022] GET /identity/v3/groups/fce1f1021a434d1b8a779932c19fcc75 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:55.200768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.272430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.273111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.273370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.273650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.274929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:55.274929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:55.276301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f12901-be49-4f50-8f2c-5d6c2573f5c9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:55.287096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 197/379] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:23:54 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=a2ae414a10a54833a1002750699b5d83 => generated 260 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:55.315473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce7db-514e-49df-b2d0-25e097009448 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:55.326298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:55.367647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.368362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.368651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.368936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.369797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] RBAC: Adding query filter params (group.id=fce1f1021a434d1b8a779932c19fcc75, role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=7486fcd7991c440786c9d72536ea0aed) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:55.370116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:55.372732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960996e4-fad7-4c1a-b591-4d9004f8f339 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:55.379325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 183/380] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:23:55 2022] GET /identity/v3/role_assignments?group.id=fce1f1021a434d1b8a779932c19fcc75&scope.project.id=7486fcd7991c440786c9d72536ea0aed&role.id=4ef72abf5489405bbc82b2318479765b => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:55.389179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e608f9-4e26-40a8-b36a-d72595ee462b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.389531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e608f9-4e26-40a8-b36a-d72595ee462b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.389736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e608f9-4e26-40a8-b36a-d72595ee462b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.393152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 198/381] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:55.400563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50e0d-d031-4477-be67-bb5679fecf89 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.400869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50e0d-d031-4477-be67-bb5679fecf89 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.401130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50e0d-d031-4477-be67-bb5679fecf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.426322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-02d50e0d-d031-4477-be67-bb5679fecf89 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:55.427710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02d50e0d-d031-4477-be67-bb5679fecf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:55.518277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 184/382] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:55.545927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e608f9-4e26-40a8-b36a-d72595ee462b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.670851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a8f7-76df-4ec6-a38a-48c4aa0e4de0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.671167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a8f7-76df-4ec6-a38a-48c4aa0e4de0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.671422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3a8f7-76df-4ec6-a38a-48c4aa0e4de0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.672937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 185/383] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:55.679939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.680502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.680991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.681786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:55.700849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e608f9-4e26-40a8-b36a-d72595ee462b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:55.702215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:55.703727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:55.762866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:55.782877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.782877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.782877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.782877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.800511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:55.815603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:55.816617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:55.817454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 199/384] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:55.823881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.843827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 186/385] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:55.882733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:55.940528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8cc89-49c2-446f-ba23-d0530bf2af16 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:55.982574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:55.983251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:55.983592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:55.983865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:55.984662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:55.984978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:55.986364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:55.997086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 200/386] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:56.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.024698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:56.080028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d6972-7e13-4701-978b-df6ffe8a75fc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.119640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a819a-0ee7-493e-adb9-66e514b4f292 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.140639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.141458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.141785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.142062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.159930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.160791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.161071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.161344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.164474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.166233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.168390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:56.171389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 201/387] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:56 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:56.180780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:56.181707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:56.182503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 187/388] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:56.184434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.191958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.242359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:56.291484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:56.309411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220f5a0-7d51-472e-87c3-89922e2b23c0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.353510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.357789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.358323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.358831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.363692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.366368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.370416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 202/389] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:56 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:56.379869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.442412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b396d-3c3d-4901-bf29-cb5146d4505a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.496305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9515f-b8e7-4b88-933d-8963c448d7e9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.531913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.532948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.533451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.533970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.534216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.535128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.535408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.535602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:56.535706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/groups/3675afe49a424b208591ecb23ff644e0/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.536263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.537881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.537983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, group_id=3675afe49a424b208591ecb23ff644e0, role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.539404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.550708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 188/390] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:56.559204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.616625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 203/391] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Jun 24 14:23:56 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/groups/3675afe49a424b208591ecb23ff644e0/roles/a2ae414a10a54833a1002750699b5d83 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:56.658282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:56.675568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27532bd-bcb7-4283-8592-dd3600b032a2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.714209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.715661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.715958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.716222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.717514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.719053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.721087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 189/392] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:56 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:56.726284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:56.755736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:56.864926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf5c1b-67b4-420d-adab-b9e08bbbffbb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:56.904945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:56.905751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:56.906035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:56.906308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:56.907573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:56.909072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe49cc-0ab1-4a8e-9a5e-cccbd98933aa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:56.911252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 204/393] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:56 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:56.916582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:57.066980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728fa0c-1553-4e38-adf7-ed177f87f56f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:57.116202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:57.117084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:57.117364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:57.117635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/groups/3675afe49a424b208591ecb23ff644e0/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:57.123814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] RBAC: Authorizing `identity:create_grant(project_id=44cdc47bad92408e9fa0cadeaa7e1270, group_id=3675afe49a424b208591ecb23ff644e0, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:57.125256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c3a8e6-29db-4171-bae7-1da4e3a5f917 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:57.222412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 190/394] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Jun 24 14:23:56 2022] PUT /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270/groups/3675afe49a424b208591ecb23ff644e0/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:57.524339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2ba4-6b3a-49e0-b4c7-e17d76ac4d2e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:57.524669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2ba4-6b3a-49e0-b4c7-e17d76ac4d2e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:57.524841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d2ba4-6b3a-49e0-b4c7-e17d76ac4d2e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:57.526498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 205/395] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:57.533244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:57.533530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:57.533806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:57.556164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:57.557702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:57.572607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:57.658786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 191/396] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:57.702244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2ba4-6b3a-49e0-b4c7-e17d76ac4d2e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:57.813732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:57.843200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d2ba4-6b3a-49e0-b4c7-e17d76ac4d2e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:57.885354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:57.886203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:57.886430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:57.887257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:57.902062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b9dac50d-75dd-4ff5-a253-b2a6aed721bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:57.910733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:57.911620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:57.912811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 206/397] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:57.918239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.060537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c384-e629-4c18-918f-54b5668fd6e0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.061187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c384-e629-4c18-918f-54b5668fd6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.061658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215684a9-6eb8-434e-bf66-7242a3456a5c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.061877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c384-e629-4c18-918f-54b5668fd6e0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.064823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 207/398] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:23:58.069408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.069998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.070494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.089695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:58.092254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:58.122489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.124002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.124517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.125074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.126134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:58.126687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:58.128405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:58.139624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 192/399] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:58.148328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.168759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:58.238717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 208/400] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:58.315765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.331217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a7149-cce1-4e48-88a0-d047ae1074f9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.400656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.401525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.401807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.402104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.403476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:58.405060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec73e11e-2ba5-434d-af79-2c73461a85d6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:58.412628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 193/401] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:58.414034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec315-f05a-41fc-bf63-0256e9b943b5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.414324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec315-f05a-41fc-bf63-0256e9b943b5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.414624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec315-f05a-41fc-bf63-0256e9b943b5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.416263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 194/402] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:58.416970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec315-f05a-41fc-bf63-0256e9b943b5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.425731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:58.465907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc231c3-9bc4-4c7c-b5f1-88c55e6d682e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.500557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.501372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.501644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.501920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.513111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:58.519184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a287b697-fe44-4564-83ba-8e8966b75745 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:58.521624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:58.522847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:58.524160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 209/403] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:23:58.524843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.525116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.525212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.551762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:58.553621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:58.562437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec315-f05a-41fc-bf63-0256e9b943b5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.603884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.604722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.605004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.606247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.607386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:58.609010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:58.615647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 195/404] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:58.616644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.636565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 210/405] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:58.638229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.706897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:58.750147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4072b-896e-49b3-a50c-d3cc4bfa52fe None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.750690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44588eb-fed5-4f73-a458-1e8233e5098b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.785472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.786125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.786410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.786764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.787517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:58.787787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:58.789393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:58.790152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.791385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.793672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.793672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/groups/3675afe49a424b208591ecb23ff644e0/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.795850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, group_id=3675afe49a424b208591ecb23ff644e0, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:58.797494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:58.802876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 196/406] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:58.804034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.807546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:58.909662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe6627-5e59-46cb-8202-5bbcee186a6a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:58.910087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 211/407] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Jun 24 14:23:58 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/groups/3675afe49a424b208591ecb23ff644e0/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:58.911622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:58.958548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:58.959383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:58.959716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:58.960000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:58.975498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:58.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:58.986560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:23:58.987805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 197/408] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:58.993915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.073841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.125023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df514d4e-683b-423f-a42e-989efd29e5a9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.150145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feed587-7efa-4b2a-8df5-cb5f043db1d4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.173866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.174875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.175480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.175942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.176704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:59.177015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.177773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.178699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225264c-5df1-4acf-aae4-a1198d040b3c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.189316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 198/409] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:23:59.190972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321649-f031-4653-b015-450ecc87fa31 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.192427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321649-f031-4653-b015-450ecc87fa31 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.192427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321649-f031-4653-b015-450ecc87fa31 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.193337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 199/410] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:59.194073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.198565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.198565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.198565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.199428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.199722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.200002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.201977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.203630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.205372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 212/411] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:58 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:59.205987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.220972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:59.222620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:59.298549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 200/412] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:23:59.301339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.332545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f63f2-d1b0-4d50-b5da-35122a939603 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.376571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.378020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.378502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.379150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] PATH_INFO: `/v3/groups/fce1f1021a434d1b8a779932c19fcc75` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.381140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] RBAC: Authorizing `identity:get_group(group_id=fce1f1021a434d1b8a779932c19fcc75)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.383613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.386338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 213/413] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/groups/fce1f1021a434d1b8a779932c19fcc75 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:59.387149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.388124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.430360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c713f4-f41b-4fed-b5fd-2a191188ce1b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.462888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.467522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.468394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.468675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.468946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.470156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.471969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.474014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 201/414] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:59.479038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.513945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e9a0d-1b52-48d8-bcda-7449532aabb2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.549581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.550384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.550698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.550972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.562309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:23:59.570405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:23:59.571332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:23:59.572136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 214/415] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:23:59.572882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.594144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263ac90-2419-4aab-bd48-fdebc5a8834d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.618464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.629203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.632879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.632879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.632879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] PATH_INFO: `/v3/projects/7486fcd7991c440786c9d72536ea0aed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.632879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] RBAC: Authorizing `identity:get_project(project_id=7486fcd7991c440786c9d72536ea0aed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.633453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3959d2f-c831-4408-a01f-96b6c72e5a73 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.635335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 202/416] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/projects/7486fcd7991c440786c9d72536ea0aed => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:59.641601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98a5b-9b80-4145-b777-150b27b2533c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.641892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98a5b-9b80-4145-b777-150b27b2533c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.642175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c98a5b-9b80-4145-b777-150b27b2533c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.643498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 203/417] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:23:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:23:59.644191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c98a5b-9b80-4145-b777-150b27b2533c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.685450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725c149-710c-4fab-a08d-0d63cd88cc70 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.704554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.721714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.722620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.722896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.723167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/groups/3675afe49a424b208591ecb23ff644e0/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.725361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] RBAC: Authorizing `identity:create_grant(project_id=dec316ea60f64c8bb28ed645fe6cd250, group_id=3675afe49a424b208591ecb23ff644e0, role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.726824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.760336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c98a5b-9b80-4145-b777-150b27b2533c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:23:59.794345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:23:59.795037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.795320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.795598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.796338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:23:59.796645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:23:59.798001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8743038-aee7-430d-8c05-8f46163b2e35 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:23:59.806102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 215/418] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Jun 24 14:23:59 2022] PUT /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250/groups/3675afe49a424b208591ecb23ff644e0/roles/a2ae414a10a54833a1002750699b5d83 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:23:59.807185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.811191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 204/419] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:23:59.814081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:23:59.814677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:23:59.815138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:23:59.839146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:23:59.840702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:23:59.905984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:23:59.931685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 205/420] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:23:59.932961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:23:59.963205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04ec09-cd24-47d3-ae45-26c2f659f4ba None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.010100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.011993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.012699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.013155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] PATH_INFO: `/v3/projects/7486fcd7991c440786c9d72536ea0aed/groups/fce1f1021a434d1b8a779932c19fcc75/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.015437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] RBAC: Authorizing `identity:create_grant(project_id=7486fcd7991c440786c9d72536ea0aed, group_id=fce1f1021a434d1b8a779932c19fcc75, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.016897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:00.056563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81167294-0e28-4f22-b9aa-6c527d6af3a1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.092554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 216/421] 10.222.0.23 () {58 vars in 1442 bytes} [Fri Jun 24 14:23:59 2022] PUT /identity/v3/projects/7486fcd7991c440786c9d72536ea0aed/groups/fce1f1021a434d1b8a779932c19fcc75/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:00.093884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.099540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.100258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.100543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.100937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.184549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.218180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.236298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.238236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:00.302570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430392fb-3403-41f0-898b-e9badcd56e6c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.342749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.343560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.343843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.344115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.367558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:00.370330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 206/422] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:00.376242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:00.376373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:00.377414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:00.378296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 217/423] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:00.383439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.470022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.487368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb43d-2781-41fa-afc3-8926416bbbb3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.530872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.531254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.532138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.532420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.532696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.537010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd44c9-29b3-4842-b926-5137a543d2da None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.539377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.540865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:00.549794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 207/424] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:00.550712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.600202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.600886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.601167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.601442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.602197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:00.602503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.603913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:00.627063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 218/425] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:00.632288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.753439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69981caa-88fb-4f39-8812-82d52783df9f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.777524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ff38b-7941-438f-b52f-d71695d41d30 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:00.815016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.815729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.816011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.816287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.831567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:00.832744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:00.833240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:00.833711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:00.835131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.836944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:00.838822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 208/426] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:00.844709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:00.852077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.891901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:00.896113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:00.897659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba35f20-0590-4914-87e4-874d6b62faa2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.030791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf9d1a-08cf-4d1a-a7c9-da91581c634f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.069876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.071024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.071318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.071602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.072775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:01.073099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:01.078538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.082320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 209/427] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=a2ae414a10a54833a1002750699b5d83 => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:01.085024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 219/428] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:24:01.091854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.209089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:01.238596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0db3-7bae-4568-8fe4-6245e871411d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.238954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0db3-7bae-4568-8fe4-6245e871411d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.239159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e0db3-7bae-4568-8fe4-6245e871411d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.239701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d7953-3948-4c3b-84ec-79c9332755a2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.240820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 220/429] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:01.247297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.247597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.247859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.271462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:01.272699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:01.306725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.307709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.308133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.308481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] PATH_INFO: `/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.309797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] RBAC: Authorizing `identity:get_project(project_id=44cdc47bad92408e9fa0cadeaa7e1270)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:01.311377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.317605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 210/430] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/projects/44cdc47bad92408e9fa0cadeaa7e1270 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:01.318700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.357366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:01.372293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 221/431] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:24:01.398659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.458205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ffe1-497a-46b5-a892-5827ed7ae71f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.503141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.504010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.504295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.504570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.506283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:01.508263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.510946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 211/432] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:01.517826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.533542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ee88-cf48-4e55-b5ef-93e76a773a9b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.545145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:01.569253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913611-0104-450d-abed-c0d832935330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.569976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913611-0104-450d-abed-c0d832935330 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.570262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913611-0104-450d-abed-c0d832935330 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.570538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913611-0104-450d-abed-c0d832935330 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.581368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41913611-0104-450d-abed-c0d832935330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-41913611-0104-450d-abed-c0d832935330 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:01.589262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:01.590667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 14:24:01.591471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:01.592159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 222/433] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:01.598609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913611-0104-450d-abed-c0d832935330 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.651264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08e29f-c2fa-4e87-8232-c5c4a87d7062 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.653129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:01.693933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.695009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.695542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.696108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.697132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:01.697673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:01.700070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.707118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 212/434] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=44cdc47bad92408e9fa0cadeaa7e1270&role.id=66ef983ebbae442b8cab404599a2d333 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:01.719725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913611-0104-450d-abed-c0d832935330 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.763153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.764640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.764640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.765135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:01.766134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:01.766690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:01.768395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:01.779114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 223/435] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:01.785134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:01.882247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:01.898807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7eb09f-d0be-4554-877a-1ba84a2a8d4b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:01.936374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:01.937098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:01.937386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:01.937662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.010648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:02.012265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364198c-406e-47f0-ad60-f9c87c8901bd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:02.100569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.118102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 213/436] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:02.123969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.242994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb8e9-840e-4651-8877-dd5ea4b1a6db None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.254927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470dd9-9c4a-4083-af72-30873a2cb83e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.255437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470dd9-9c4a-4083-af72-30873a2cb83e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.255515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470dd9-9c4a-4083-af72-30873a2cb83e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.256995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 214/437] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:02.257561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.264616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.265383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.265683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.281918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.283149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.283733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.284234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.284526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:02.285725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:02.286093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:02.288779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:02.291103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 224/438] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:02.302506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.383063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 215/439] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:02.406764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.431837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e08b8-4502-401e-9ffb-09ba6f4e5f36 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.469200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.470071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.470364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.470675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.471866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:02.473317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f8020-6935-423d-9478-40573ed8ca9d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:02.474977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 225/440] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:02.476257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84cf0-f250-4f17-a6aa-815ac1b25c02 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.476574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84cf0-f250-4f17-a6aa-815ac1b25c02 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.476861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84cf0-f250-4f17-a6aa-815ac1b25c02 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.478103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 226/441] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:02.481199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd84cf0-f250-4f17-a6aa-815ac1b25c02 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.509445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08cea1-4f23-480d-ac8a-a814c2fd0e67 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.524477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.534219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.547996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.548800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.549082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.549360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.560093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b8a82bfe-12ea-4cf0-a20c-f8a8c7d24919 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:02.568551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:02.569726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:02.570609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 216/442] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:02.573061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.573582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.574356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.587138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd84cf0-f250-4f17-a6aa-815ac1b25c02 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.591462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:02.592669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:02.621258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.621912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.622194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.622470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.623321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=66ef983ebbae442b8cab404599a2d333, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:02.623632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:02.625060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:02.630786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 227/443] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=66ef983ebbae442b8cab404599a2d333 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:02.631561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.664420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 217/444] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:02.688765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.762097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b00190-276d-40f8-a814-aa9664237ba2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.776304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.802112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.802310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.802586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.803017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.803811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:02.804115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:02.805566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:02.811670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1d79a-c6d9-4378-842a-c0716ae03574 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.814982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:02.818599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 228/445] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:02.820587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.856467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.857955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.858441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.858957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:02.871476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:02.879296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:02.880208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:02.880997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 218/446] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:02.886906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:02.942156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf9281-31cc-4977-9ebc-bf9df772ae33 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:02.987321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:02.987993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:02.988280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:02.988551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.003758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c72d26-5754-4646-add5-f0d362d302c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.033509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.037432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.038099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.038384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.038717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.039470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:03.039804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.041265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.049398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 219/447] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:03.055198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.059490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.060964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.083519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.155886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91942198-c9b9-41a2-bce2-e181257ba757 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.158209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 229/448] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:24:03.163903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.188867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.189516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.189799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.190072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.238162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.257599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.259110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.263861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bfd5e2-7d82-4680-973b-1685ce4e3a9b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.286378 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.298722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.299629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.299910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.300188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] PATH_INFO: `/v3/projects/dec316ea60f64c8bb28ed645fe6cd250` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.301379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] RBAC: Authorizing `identity:get_project(project_id=dec316ea60f64c8bb28ed645fe6cd250)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.303144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.304845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 230/449] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/projects/dec316ea60f64c8bb28ed645fe6cd250 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:03.310237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.360794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 220/450] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 14:24:03.366270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.416520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e529f4-da10-4eb1-8029-9801780097aa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.449332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.450295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.450645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.450923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] PATH_INFO: `/v3/groups/3675afe49a424b208591ecb23ff644e0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.452100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] RBAC: Authorizing `identity:get_group(group_id=3675afe49a424b208591ecb23ff644e0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.453539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.455190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 231/451] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/groups/3675afe49a424b208591ecb23ff644e0 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:03.460851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.467030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76bdb5-c102-4e62-9311-14b10c78c822 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.473213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.492334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.501354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.502211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.502493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.502798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] PATH_INFO: `/v3/projects/7486fcd7991c440786c9d72536ea0aed` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.503990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] RBAC: Authorizing `identity:get_project(project_id=7486fcd7991c440786c9d72536ea0aed)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.505474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.507194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/projects/7486fcd7991c440786c9d72536ea0aed => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:03.513270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.566069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67fba6-dc64-4452-abba-4ae005487a4e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.590381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.591067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.591329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.591599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.592415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] RBAC: Adding query filter params (group.id=3675afe49a424b208591ecb23ff644e0, role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=dec316ea60f64c8bb28ed645fe6cd250) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:03.592720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.594147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.599834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/role_assignments?group.id=3675afe49a424b208591ecb23ff644e0&scope.project.id=dec316ea60f64c8bb28ed645fe6cd250&role.id=a2ae414a10a54833a1002750699b5d83 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:03.609576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad3b3a-409c-406e-8a09-99d2cad0663f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.647385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.648269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.648547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.648817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] PATH_INFO: `/v3/groups/fce1f1021a434d1b8a779932c19fcc75` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.650071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] RBAC: Authorizing `identity:get_group(group_id=fce1f1021a434d1b8a779932c19fcc75)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.651596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a758-a68a-417e-a1f8-427d73d38e98 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.653549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/groups/fce1f1021a434d1b8a779932c19fcc75 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:03.659560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:03.738864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:03.764172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f22ca4-aca4-4e64-9f0c-823be46b4430 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:03.798409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:03.799090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:03.799373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:03.799644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:03.800461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] RBAC: Adding query filter params (group.id=fce1f1021a434d1b8a779932c19fcc75, role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=7486fcd7991c440786c9d72536ea0aed) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:03.800773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:03.802212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01a9c-9cf7-42c4-b169-72e48f5799a1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:03.809194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Fri Jun 24 14:24:03 2022] GET /identity/v3/role_assignments?group.id=fce1f1021a434d1b8a779932c19fcc75&scope.project.id=7486fcd7991c440786c9d72536ea0aed&role.id=4ef72abf5489405bbc82b2318479765b => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:06.255880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650990-fb72-43e1-b0ef-338d646fb4ce None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.256884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650990-fb72-43e1-b0ef-338d646fb4ce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.257395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7650990-fb72-43e1-b0ef-338d646fb4ce None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.259217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:06.266355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf41c6-dddc-4bc8-ac7f-a23ff7534df6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.266883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf41c6-dddc-4bc8-ac7f-a23ff7534df6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.267391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf41c6-dddc-4bc8-ac7f-a23ff7534df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.306047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e0bf41c6-dddc-4bc8-ac7f-a23ff7534df6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:06.308131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e0bf41c6-dddc-4bc8-ac7f-a23ff7534df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:06.470651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:06.508419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650990-fb72-43e1-b0ef-338d646fb4ce None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:06.512086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4c23-489c-4ef6-af3a-2dbb45f75c59 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.513164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4c23-489c-4ef6-af3a-2dbb45f75c59 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.516932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e4c23-489c-4ef6-af3a-2dbb45f75c59 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.526623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 235/458] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:06 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:06.540431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab972-938b-4818-81c1-6457f253626c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.540778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab972-938b-4818-81c1-6457f253626c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.541062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ab972-938b-4818-81c1-6457f253626c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.574120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-330ab972-938b-4818-81c1-6457f253626c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:06.575343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-330ab972-938b-4818-81c1-6457f253626c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:06.659036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:06.687737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:06.694336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 236/459] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:24:06.719975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7650990-fb72-43e1-b0ef-338d646fb4ce None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:06.756661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:06.757728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.758154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.758644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.770366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:06.772026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:06.775019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ab972-938b-4818-81c1-6457f253626c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:06.806329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:06.807411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:06.808376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:06.808376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:06.808376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:06.808376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:06.810542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 224/460] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:06.816761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:06.938695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ceadd-be01-4940-8a5e-445c8d9c9f97 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:06.955889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:06.978830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:06.979882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:06.980317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:06.980735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:06.981604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:06.982066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:06.983679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:06.997270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 225/461] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:07.004217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.059685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ab972-938b-4818-81c1-6457f253626c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.100128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.101252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.101537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.101817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.113774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.115295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.127570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33c20a-4113-4102-88fc-6f7d8277a098 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ba8c807e-7e43-4646-92fb-433d47b2db94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:07.137956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:07.138988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 237/462] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f9-be97-4ac5-8a98-bad11e7ac360 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f9-be97-4ac5-8a98-bad11e7ac360 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.144627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d885f9-be97-4ac5-8a98-bad11e7ac360 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.146131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 238/463] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:07.147160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23598-a3da-4653-89cd-d0447fe1539e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.147445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23598-a3da-4653-89cd-d0447fe1539e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.147720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c23598-a3da-4653-89cd-d0447fe1539e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.153558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 239/464] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:07.153558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c23598-a3da-4653-89cd-d0447fe1539e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.176164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.176841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.177126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.177693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.178381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.179855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b63b92-8da5-4879-a22f-b080325540ea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.203457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:07.254074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c23598-a3da-4653-89cd-d0447fe1539e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.294596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 226/465] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:07 2022] POST /identity/v3/users => generated 246 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:07.294596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.294596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.294596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.298237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.298966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.299249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.299521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.300400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:07.300705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.302232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2081e10-5908-43d1-9b95-807399b56171 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.314706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:07.315796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 240/466] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:07.317171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.317459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.317734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.320883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:07.337863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:07.339737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:07.347610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:07.425912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 227/467] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:07.427068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.444733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 241/468] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:07.458286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.512219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:07.567454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dede60-84e4-4024-bd3b-b62eee8f6e1b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.586218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f3e77-3f4c-4123-adb2-9026acd51c06 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.606197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.606877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.607156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.607431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.608133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.609571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.624362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.625248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.625629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.625981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.637087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.638617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:07.658088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:07.659103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:07.659953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:07.659953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:07.659953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.659953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:07.659953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 242/469] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:07.660653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.717111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 228/470] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:07 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:07.723400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.771071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:07.812232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:07.828513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c8470-bce9-4c31-9833-0eccd43eaf60 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.907194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.908397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.908906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.909393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.917418 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8685f-545f-4e5b-9760-5d3208b20202 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:07.929770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.931743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.958908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:07.959263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:07.960079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:07.961022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:07.961022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:07.961022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:07.961022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:07.961022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 243/471] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:07.962483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:07.962980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:07.963525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:07.965883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:07.966428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:07.967242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:07.968331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:07.983074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 229/472] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:07.989503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:08.083432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:08.100294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33505c2d-2cbd-4db9-a619-099029f387d2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:08.116041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc9fc6-c6a4-432b-b93e-1937669d977d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:08.139372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:08.140049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.140428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.140710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.141440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:08.141748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:08.143261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aec980-3c81-4f82-abf0-c8e28b86268f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:08.156570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 244/473] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:08.157915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541eb3-0941-4034-8033-aad0ab00e49c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.158310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541eb3-0941-4034-8033-aad0ab00e49c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.158698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2541eb3-0941-4034-8033-aad0ab00e49c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.159708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:08.160188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 245/474] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:08.160378 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.160659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.160932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.161634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:08.163200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43dd1a0-6ab0-4521-91ed-44994decf8ee None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:08.165034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2541eb3-0941-4034-8033-aad0ab00e49c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:08.169842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:08.272357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2541eb3-0941-4034-8033-aad0ab00e49c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:08.286494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 230/475] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:07 2022] POST /identity/v3/users => generated 249 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:08.288605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.289088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.289772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.314596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:08.317011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:08.324990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:08.325769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.326050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.326326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.327119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:08.328770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:08.414113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 231/476] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:08.420444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 246/477] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:08 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:08.453882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:08.544825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:08.588532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c2034-aac6-40ac-8aaf-072572e401e5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:08.629757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:08.630802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.631285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.631774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.632734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:08.634303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:08.685399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.sql.core [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 14:24:08.685399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 14:24:08.685399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [parameters: {'id': 'ad94650040f747ccbe76609014120a34', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 14:24:08.685399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80353) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 24 14:24:08.702835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:24:08.707225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application yield resource Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise self.value Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application session.commit() Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 14:24:08.708127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.flush() Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application compat.raise_( Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 14:24:08.708987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:24:08.709844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:24:08.711015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [parameters: {'id': 'ad94650040f747ccbe76609014120a34', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:08.712829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 14:24:08.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:08.717245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 232/478] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Jun 24 14:24:08 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:08.717378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:08.794081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:08.849303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27694d9-7cd7-49d5-9fd1-c5316ca47e05 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:08.901008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:08.902769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:08.903053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:08.903330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:08.916489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9d76e310-afdf-4160-83fa-ad97dd121ad8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:08.925366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:08.930184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:08.931023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 247/479] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:08.932702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:09.092668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc2149-73bc-4cae-ab41-6218fba86873 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:09.119554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:09.128928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:09.129598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:09.129881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:09.133996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:09.133996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:09.133996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:09.133996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df3807-040d-4de2-9def-7cd5474390e3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:09.149050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 233/480] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:09.671693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51721f70-dec1-428e-85e3-12e5745b9577 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:09.672178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51721f70-dec1-428e-85e3-12e5745b9577 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:09.672178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51721f70-dec1-428e-85e3-12e5745b9577 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:09.673803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 248/481] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:09.679324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:09.679744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:09.679914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:09.701125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:09.702312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:09.780576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 234/482] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:09.815513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51721f70-dec1-428e-85e3-12e5745b9577 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:09.960286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51721f70-dec1-428e-85e3-12e5745b9577 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:09.984545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:10.006964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:10.007761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.008038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.008309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.029515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c337f111-1eda-4965-9db9-0033b8f4a834 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:10.037427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:10.038299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:10.039158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 249/483] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:10.046966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:10.223234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694723-2a1b-4b96-aff4-68cb44a693ab None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.223576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694723-2a1b-4b96-aff4-68cb44a693ab None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.223745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e694723-2a1b-4b96-aff4-68cb44a693ab None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.225279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 250/484] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:10.230746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.231034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.231313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.263424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:10.264610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:10.273009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9bb3f-81fe-40e9-ace2-65aebba13138 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:10.310894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:10.326375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:10.327070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.327350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.327618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.328364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:10.328670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:10.330050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:10.338664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 235/485] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:10.343881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:10.413095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 251/486] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:24:10.474373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:10.497381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2863a-5366-464a-acf4-56b5fc4cb89c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:10.531713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:10.532510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.532788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.533057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.544072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:10.545542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:10.564122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:10.565009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:10.565847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:10.565847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:10.565847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:10.565847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:10.565847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 236/487] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:10.577179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:10.642880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:10.701752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c4001-8249-45e4-92f8-e3907177e259 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:10.735047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:10.774882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:10.777203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.780488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912c2da-38e4-45a6-907b-39006c494be3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:10.781331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.781778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.804788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:10.813025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:10.814118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:10.815236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:10.829113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 252/488] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:10.834774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:10.851317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:10.852006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:10.852291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:10.852567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:10.853370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:10.853682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:10.855928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:10.881676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 237/489] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:10.887786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.004239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e63a-d6e5-4d2d-9019-7a6f7d9f6791 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.024623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.034796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb72d8e7-cd05-402c-a626-0c68e961d5e9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.051444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.052389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.052685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.052963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.053884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:11.054196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.055924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61e017-193c-4e3b-87fd-c11b38c8fdd7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.066598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 253/490] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:11.068223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6295b-1637-41fb-b110-9750a9b74208 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.068708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6295b-1637-41fb-b110-9750a9b74208 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.069175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6295b-1637-41fb-b110-9750a9b74208 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.075164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 254/491] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:11.076369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6295b-1637-41fb-b110-9750a9b74208 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.078553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.079406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.079694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.079973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.094548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.094745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.117653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a42ef764-5bda-4a8e-9d35-cf5210cd64fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:11.120725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:11.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:11.122962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:11.125183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 238/492] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:11.127414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.127843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.138442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.147097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:11.148295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:11.198069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6295b-1637-41fb-b110-9750a9b74208 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.222836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 239/493] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:11.223499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.244395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.246291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.246771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.247241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.262252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.264697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-14b0fb1b-93b1-437e-8b5b-f09bd24d3f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:11.285173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:11.286042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:11.287368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:11.287368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:11.287368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:11.287368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:11.287368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 255/494] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:11.287638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70fb8e-a541-4a67-8b27-306caeb635b1 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.287900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70fb8e-a541-4a67-8b27-306caeb635b1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.288175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70fb8e-a541-4a67-8b27-306caeb635b1 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.289582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 256/495] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:11.290130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70fb8e-a541-4a67-8b27-306caeb635b1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.319489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.339416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab24fc5-fa7c-4016-b7e0-53207094ee14 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.378333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.379060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.379343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.380024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.380876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:11.381188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.383338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.389480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.405985 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70fb8e-a541-4a67-8b27-306caeb635b1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.427704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 240/496] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:11.427704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.445951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.446769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.447053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.447325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.458155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-012fe516-e6a3-4366-b5a6-7114b967c0c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:11.466531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:11.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:11.468277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 257/497] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:11.468953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.469541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.469541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.486246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:11.487455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:11.537001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b41631-dd73-481f-8455-0f6e739c6e83 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.562550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 258/498] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:11.562550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.584627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.585400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.585687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.585959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.586711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:11.587017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.588412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.597970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.606397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 241/499] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:11.606397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.643307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.668414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b3731-96e0-4682-bd27-f6b5787abd9b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.705060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.705824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.706108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.706390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.711415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d4353-0806-4628-9dba-d7b5e453d0af None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.746041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.746767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.747057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.747336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.748085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:11.748394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.749784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.762813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 242/500] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:11.762813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.784634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:11.790090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:11.820152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.860529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30a31-b365-41ac-809a-515c7d0b49d5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:11.868473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:11.894723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 259/501] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:11.895471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.896513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:11.897311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:11.897763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:11.898042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:11.909139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:11.916872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:11.917947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 243/502] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:11.919207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:11.997851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51d0b-a8ae-4f4f-9c46-b96b8a082bbf None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.020966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c300-3cb0-4108-b50c-f6f42d4d504f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.031547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.032337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.032618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.032888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.039787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.041282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.042788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.053853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.057892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.057892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.057892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-28e71bf6-caa8-4815-9f24-b9e45be07b6d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:12.058222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.059204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:12.060113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 260/503] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.061052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8c1e0-e553-4bca-8ee8-6743c147fbed None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.061052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8c1e0-e553-4bca-8ee8-6743c147fbed None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.061294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e8c1e0-e553-4bca-8ee8-6743c147fbed None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.063194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 261/504] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:12.064181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8c1e0-e553-4bca-8ee8-6743c147fbed None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.066841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.068306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:12.086736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.087739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 244/505] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.088862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.103597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.172545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e8c1e0-e553-4bca-8ee8-6743c147fbed None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.189448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53623e2-1ead-4de2-820f-f492a5e4c399 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.206658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.207441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.207723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.207998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.218143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.219622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.222380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.223075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.223362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.223642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.224384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:12.224694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.226076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.234364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 245/506] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:12.235177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-92d8f67b-9526-4efe-92be-8318902b2d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:12.237327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.238146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.239053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:12.239053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:12.239053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.239053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:12.239053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 262/507] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.240213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.240503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.240784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.257286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:12.258478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:12.277098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.324484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 263/508] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:12.325322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.344217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d99c2-7ae3-44e4-944d-0b2475f9fe0c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.344976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.381197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.381868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.382148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.382422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.383286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:12.383597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.385038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.402207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 246/509] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:12.403039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.431429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fa644-4877-4ac4-9dd9-e811573bb896 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.466524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.467202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.467481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.467758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.468481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:12.468782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.470170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.482075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 264/510] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:12.482925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.499677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.505072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb732aa9-a39d-4116-b6c9-717c01385893 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.538776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.539558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.539837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.540100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.551009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.552712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.558989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:12.569974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.576393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 247/511] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.577259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.583997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc7b63-df82-4b86-9e33-010c1bda09a1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.617695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.618353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.618669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.618945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.619691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:12.620001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.621358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.632699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 265/512] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:12.633478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.670917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dd3d8-065e-422b-9fa3-f493afe8916d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.703599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.704402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.704670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.704947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.715457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.716911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.723716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.730731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b0af7-302d-430b-8f9b-fe340b87123d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:12.741079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.741968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 248/513] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.742860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.764547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.765190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.775030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.775030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.775030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.832316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.833968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.836816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff0418-4e4a-403e-8a93-68fa84974e60 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:12.871427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:12.872262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:12.872547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:12.872831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:12.879901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:12.881341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:12.896278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:12.897380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:12.898298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 249/514] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:12.898900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.935948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 266/515] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:12.936790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:12.958870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:12.970266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.001405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07cce8-b98d-4f51-9c85-89288723de57 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.034204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255677-54cc-4e4d-92c2-77271d171138 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.034639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.035333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.035617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.035893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.036618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:13.036918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.038302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.050304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 250/516] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:13.051125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.071691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.072482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.072891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.073249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.079727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.081160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:13.095208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:13.096214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:13.097065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 267/517] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:13.097539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.154537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264267b0-c791-492d-99fd-35855c877fe9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.171446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.188751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.192753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.192753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.192753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.192753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:13.192753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.193278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.193352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.196984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a43d8-a5ed-4751-8a5e-4a8d381cc52f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.204183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 251/518] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:13.204979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.231189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.231855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.232135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.232411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.233236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:13.233543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.235027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.271835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 268/519] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:13.272996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.305260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f41c-c973-4970-a5d7-63a5e1a8997d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.339248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.340037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.340324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.340605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.350915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.352303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.370817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e618501-549a-4f01-bb9e-729d95abcfe4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:13.372796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:13.373816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 252/520] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:13.375741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.404863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.407392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.408189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.408469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.408743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.415768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.417393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:13.432568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:13.433453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:13.434344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 269/521] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:13.435236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.450587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.477375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809d21a-2e11-44fe-9a2f-6244c6d705d2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.513741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.514399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.514710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.515060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.516138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:13.516138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.517613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.533074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1135928-935d-4fde-910b-a06a180ba411 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.543792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 253/522] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:13.543792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.571249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.571922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.572202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.572482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.573756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.575214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.631824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.641003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608a285-8ea8-4915-9ff8-a3c1dc2d7240 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.673358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.673998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.674276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.674579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.676781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 270/523] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:13 2022] POST /identity/v3/users => generated 247 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:13.677885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.754196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.757357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.758956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.787200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e4109-8825-442d-8949-d3375522a990 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.827361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.828035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.828314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:13.828591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:13.829325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:13.829633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:13.831136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:13.844518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 271/524] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:13.845498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.863702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.876458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 254/525] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:13.877789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:13.960824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437ba97-2ed0-4661-ae91-07075da5d0c4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.991109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:13.991632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f08d9-0372-4550-b274-362331852200 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:13.997605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:13.998340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:13.998651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.002121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.002121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:14.002121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.002121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.023353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 272/526] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:14.023353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.034140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.034831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.035117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.035394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.036343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:14.036760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.038271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.050802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 255/527] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:13 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:14.050802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.112127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.164011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f995e00-758d-4a7f-824a-43220547c410 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.171950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c41b9-9018-40c9-b958-76ebfcfe05b2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.215920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.217145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.217681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.218267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.222665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.223476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.223759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.224035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.233267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.233432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.236323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.236433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:14.255575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:14.256543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:14.257399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:14.257399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:14.257399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:14.257399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:14.257639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:14.258381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 273/529] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:14.259266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.266043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 256/529] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:14.267019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.321813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.382646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c658-5853-4881-be61-9948cf5b4146 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.389217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b22b5-81f8-4c96-94bf-4bc8ab84c347 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.391810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.420642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.421579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.421712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.422056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.422887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:14.423258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.424897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.436852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.437810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.438270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.438749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.443501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 257/530] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:14.444234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.535221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.537844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.556369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247182a-72e4-4902-92a1-cfd4715e72d9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.599787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.600911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.601654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.602133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.603743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:14.604515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.606327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.610859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.624582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 258/531] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:14.625873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.655330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.659543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 274/532] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:14.660662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.737077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf8eb0-74ae-45c4-af7f-f739ca55a50e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.768751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383a928-fd5d-41af-aeb0-6e1ea9d6f542 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:14.777195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.778114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.778494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.778903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.786767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.788492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:14.806423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:14.807999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:14.809172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:14.812828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 259/533] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:14.813895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.814945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:14.815817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:14.816272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:14.816739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:14.818815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=bb628c1d9c854cbca433d36d143374c7) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:14.819373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:14.821473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:14.832477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 275/534] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/role_assignments?user.id=bb628c1d9c854cbca433d36d143374c7&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:14.833674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:14.866948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.954943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:14.985156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90e5fb-1631-44ac-b315-8651ee744b16 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.026867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72376dd3-089b-4118-9be7-be3c86cc1e60 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.070429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.071282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.071562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.071830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.094835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.098885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.100671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.101729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.102207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.102786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.104925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:15.105468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.107711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:15.118376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:15.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:15.120437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:15.120437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:15.120437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:15.120437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:15.121170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 260/535] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:15.127190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:15.140805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 276/536] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:15.157116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:15.253326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89506d-b176-460f-9300-7f9bf1e19511 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.287720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:15.305211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.305894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.306180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.306454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.307276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:15.307582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.309018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.331029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 261/537] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:15.332016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:15.334279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:15.356061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449cb51a-81df-46f1-8469-e68870b0546a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.414902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.417286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.417863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.419081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.420350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=d08dcd615d87450d9ccdc58fe89f4014) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:15.423148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.425241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.442253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 277/538] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:15 2022] GET /identity/v3/role_assignments?user.id=d08dcd615d87450d9ccdc58fe89f4014&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:15.523231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8de9a-4a8d-4cac-b056-e184cd335f0a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.581025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.582379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.582921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.583485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.592364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.594296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:15.614200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:15.619980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:15.621227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:15.623976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 262/539] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:15.629315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:15.744993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d1a7e-26ba-4736-98da-e0123fa94903 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.785736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.787307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.787791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.788306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.789306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:15.789831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.792228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a76031-2a37-4bcc-b480-f944e2d8f0db None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.795550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:15.813088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 278/540] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:15.818469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:15.884147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:15.928616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e3ebb-3416-440b-bb85-f760f6b94fe8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:15.976576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:15.977347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:15.977508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:15.977785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:15.978639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=c42ba8788bf2431a96b2087b287d7426) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:15.979791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:15.981324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2ed4e-54e7-4a8a-bb6a-7ee0c94f645e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:15.995234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 263/541] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:15 2022] GET /identity/v3/role_assignments?user.id=c42ba8788bf2431a96b2087b287d7426&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:16.187028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0957a-108c-49f1-8544-d6407317b80f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.187370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0957a-108c-49f1-8544-d6407317b80f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.187576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0957a-108c-49f1-8544-d6407317b80f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.189375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 279/542] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:16.194660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.194947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.195228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.221165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:16.222362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:16.317018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 264/543] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:16.352237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1103f-f1ed-49f8-9803-318f21a82af9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.352570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1103f-f1ed-49f8-9803-318f21a82af9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.352798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1103f-f1ed-49f8-9803-318f21a82af9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.360713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 280/544] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:16.368804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:16.371767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.372057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.372330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.401676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:16.402904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:16.471018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:16.547039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 281/545] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:16.570266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:16.584281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a18e6-888c-4f8f-90f2-e722c3e8e1d1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:16.644209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:16.645021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.645305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.645578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.664397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:16.672518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:16.673407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:16.674217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:16.675454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 265/546] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:16.695502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:16.712372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c90a-0362-4d02-ae59-691d05b44656 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:16.740097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:16.754850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:16.755654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:16.755934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:16.756204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:16.780229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:16.788124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:16.789439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:16.790301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:16.792005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 282/547] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:16.799438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.008387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410be0e5-f571-4bd7-ad6b-1d2276a19372 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.034651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.076565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.077237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.077760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.077760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.078538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:17.078846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.080220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.080535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea413-d8f7-4f64-aba0-9aeaba05a553 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.098069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 266/548] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:17.103886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.116903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.117895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.118368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.118974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.120467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:17.121026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.122834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.132042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 283/549] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:17.138948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.191717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.261592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426454e2-3c11-462d-9a2a-6840c4e40913 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.303474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c04e-3510-4b42-8a9c-b76fa9de864b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.340235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.341050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.341336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.341612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.347215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.348335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.348812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.349301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.371685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.372590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.373162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.374380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:17.393450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:17.394548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:17.395631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:17.395631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:17.395631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:17.395631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:17.397533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 267/550] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:17.403480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c8130702-fd66-46bc-830d-38e2b2fd827b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:17.414157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:17.415328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:17.416424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:17.416424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:17.416424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:17.416424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:17.424130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 284/551] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:17.425836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d3e45-b31d-49f6-a43b-c077371754be None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.428106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d3e45-b31d-49f6-a43b-c077371754be None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.428529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d3e45-b31d-49f6-a43b-c077371754be None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.431192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 285/552] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:17.431961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d3e45-b31d-49f6-a43b-c077371754be None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.445734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.523408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700915ee-a09a-46aa-bc1e-39b5619ba801 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.539744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.544484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d3e45-b31d-49f6-a43b-c077371754be None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.560732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.561716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.562237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.562711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.563602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:17.564077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.565680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9bd5f3-b1c2-4d2c-bb36-3076aa26afb8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.579611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 268/553] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:17.581342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.581789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.582251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.586138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.586913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.587262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.587603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.589013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:17.590435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.591920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.602105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:17.603607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:17.609003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 286/554] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:17.609861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.674802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 269/555] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:17.676060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.694779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.715851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d51e4-2505-4d33-956d-d9f33e8a4bb1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.751932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.752908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.753089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.753483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.762147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.763720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:17.779227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:17.780148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:17.781084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 287/556] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:17.781749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.789242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d3461-bc1c-43cb-8a56-fc86d4138998 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.811063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.832059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.833235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.833712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.834202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.843071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:17.844892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:17.860910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:17.861907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:17.863345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:17.866136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 270/557] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:17.867292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.899133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1546c8-b860-4c3c-b605-7cf221d368b7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:17.935953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:17.936777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:17.937145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:17.937318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:17.948230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:17.952116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:17.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:17.956937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:17.957754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 288/558] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:17.958563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:17.980617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b868-2ff1-4415-a28d-dc6dff27cb07 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.016901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.017968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.018255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.018567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.019541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:18.019851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.021287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.042855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 271/559] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:18.042855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.047162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.068661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce543b20-f7d5-4e65-8567-34b5b1d5d6c4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.109370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.110035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.110317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.110620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.111478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:18.111764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.113401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ba79c-1312-4c36-becb-f5bd1651b2d8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 289/560] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aaec-bad6-4643-a4ea-3513c1c3417a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aaec-bad6-4643-a4ea-3513c1c3417a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568aaec-bad6-4643-a4ea-3513c1c3417a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 290/561] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:18.135492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568aaec-bad6-4643-a4ea-3513c1c3417a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.156454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93be738-bac8-46a0-abfe-ca04b5669afe None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.193635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.194328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.194626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.194909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.195662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:18.195979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.197910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.199549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.206547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 272/562] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:18.207418 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.260846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568aaec-bad6-4643-a4ea-3513c1c3417a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.298682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.299415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.299524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.299779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.310717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.325537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fa2ee-9bba-46f1-b62e-b48594e57d9f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.358957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.359757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.360038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.360312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.371176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.371764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.372652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.373221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7ea30704-5b5a-4eff-82a9-22df4ea48aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:18.390188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:18.391302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:18.392340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:18.392340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:18.392340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.392340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:18.392340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 273/563] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:18.393013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a4ba6-e1d7-4b47-90a0-78f31b01f6f4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.393300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a4ba6-e1d7-4b47-90a0-78f31b01f6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.393573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a4ba6-e1d7-4b47-90a0-78f31b01f6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.411390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5e0a4ba6-e1d7-4b47-90a0-78f31b01f6f4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:18.412616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5e0a4ba6-e1d7-4b47-90a0-78f31b01f6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:18.455557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.474702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 291/564] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:18.475545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.481553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 274/565] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:18.483206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513983-5f80-4145-ae52-a1c500c57cc1 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.483670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513983-5f80-4145-ae52-a1c500c57cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.483826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513983-5f80-4145-ae52-a1c500c57cc1 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.485078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 275/566] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:18.485773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0513983-5f80-4145-ae52-a1c500c57cc1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.532512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.591609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0513983-5f80-4145-ae52-a1c500c57cc1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.595440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd6dcd-3808-4856-9c36-9b12b9025280 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.624964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.625619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.625898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.626168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.626915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:18.627220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.628603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.628966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.629743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.630021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.630287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.640234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 276/567] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:18.640697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.641223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.642135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-62ab3f4a-d2c2-48ca-8471-5a83d423a32c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:18.660695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:18.661813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:18.662998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:18.662998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:18.662998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.662998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:18.662998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 292/568] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:18.663348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.663615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.663907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.679676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:18.680860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:18.684098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.742988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4802a8-123b-487e-8579-7bdf56cbbdf4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.746056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 293/569] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:18.746911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.768527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.786186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.787018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.787297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.787569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.797926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.799413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:18.816636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:18.817719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 277/570] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:18.819003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.889165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa048a9-1b5f-4290-b0a8-08bb1dc1ebe6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.910978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:18.921104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb8e1e-8f78-41cb-afca-db65d1268216 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:18.921957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.922786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.923068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.923354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.933524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:18.941276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:18.942352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 294/571] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:18.943240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:18.957377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:18.958348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:18.958972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:18.959616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:18.967006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:18.968448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:18.983380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:18.984536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:18.985467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 278/572] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:18.985974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.029178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.046843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8323b76b-2c42-4dab-a78a-741ab54a2358 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.080739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.081403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.081689 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.081970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.082059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11024430-1681-4da5-b5a2-c3b34bbd746f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.082746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.083089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.084486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.096135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 295/573] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.097026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.120677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.121463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.121752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.122028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.132342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:19.140661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:19.141718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:19.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 279/574] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:19.143456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.155936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.200849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0550-9137-4ad8-9703-d8363a2699e2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.233195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.233851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.234130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.234402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.235149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.235457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.236823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.243300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee13954-003b-4be7-b0c3-33e7dc4193b4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.245905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.249722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 296/575] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.250591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.280686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.281353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.281633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.281906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.282714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.283023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.284386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.292452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 280/576] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.293169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.352414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda4662-329c-4160-b870-5b3d9bf07250 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.368677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.390851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.391451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.391733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.392008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.392786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.393095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.394797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.398994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7a6ae-ee8e-409c-a5d3-8a026446f4d0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.406122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 297/577] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.406955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.434470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.435299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.435580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.435852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.443219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.444655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:19.458839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:19.459974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:19.461128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 281/578] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:19.462696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.468665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.508905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f59b5-f643-4516-96d5-8ad174dbf175 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.541738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.542407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.542719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.542998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.543745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.544049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.545408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.553379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 298/579] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.554154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.565993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23560aea-26fa-490b-9b86-6ed657bb1e48 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.596113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.600977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.601780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.602090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.602365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.609696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.611185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:19.625865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:19.626971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:19.628134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:19.630738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 282/580] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:19.631926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.655294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7ccf5-aa15-4ba6-831e-75cc33f2b13f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.677210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.688975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.689766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.690045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.690314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.700644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.702146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:19.719198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:19.720262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 299/581] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:19.721341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.734947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e216337-9375-4991-89ec-7ef986c26a5f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.770687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.771582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.771867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.772140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.786817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=4bfc33870df145a4ba481afd9cdab92b, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.788262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.819451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92c243-1268-4a98-8e1c-6dabf00166e6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:19.852303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:19.852959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:19.853246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:19.854108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:19.855207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:19.855524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:19.856961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:19.873003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 283/582] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:19 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:19.873957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.879106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 300/583] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:19.879106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:19.927046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.928772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:19.992085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e5bdf-c52e-48f3-8846-a9df1c391f2f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.009855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44714042-4881-43a8-a383-5bff6791d6c7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.035897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.036105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.036390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.036681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.048555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.050063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.051843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.052502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.052778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.053049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.053821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:20.054129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.055844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.068376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 301/584] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:20.069228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:20.073602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:20.074470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:20.075405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:20.075405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:20.075405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:20.075405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:20.075405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 284/585] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:20.078561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.177345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.182666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.185540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5003a-d25a-4dc9-bc04-f1d0a21a1695 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.191551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335c150-8f42-43d6-8cfd-9c4c5b90950c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.222754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.223916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.224398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.224869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.227236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.229943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.230236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.230535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.231274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:20.231591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.232987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.239805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.241623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.248632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 302/586] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:20.249485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:20.268727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:20.269750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:20.270880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:20.270880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:20.270880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:20.270880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:20.271342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 285/587] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:20.272788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.374676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d263-dbf4-4b4b-8258-222b7a395a0b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.399680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942b7d-b594-40bf-8e97-613859443091 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.412100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.413010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.413010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.413304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.414125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=52486f194961487ba574eebe49869a96) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:20.414436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.416677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.422315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 303/588] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/role_assignments?user.id=52486f194961487ba574eebe49869a96&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:20.423138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.448013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.448991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.449470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.449925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.450904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:20.451315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.452828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.455317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.470123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 286/589] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:20.470123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.480437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.581661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7276784-84a3-4fe9-8e40-eb878e5dfd14 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.602006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677cd9d9-c0ab-4563-931a-8af3f0377b9b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.625893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.630628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.630628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.630628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.641749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.643628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.645550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.646530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.646935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.647837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.648728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:20.649138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.650980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:20.663375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:20.664510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:20.665666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:20.668481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 304/591] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:20.669449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.670596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 287/591] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:20.673414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.747118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.754233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:20.779016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2aa20-523a-4d2d-b8a8-1b54d4b96954 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.785908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde2fe9-049f-45cd-b15c-f2e5c8e93071 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:20.863529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.864049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:20.864855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.864990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:20.865269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.865617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.865728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:20.866174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:20.875439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.875840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:20.877073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.877661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:20.894146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:20.895095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:20.896224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:20.897071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:20.898346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:20.899315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:20.901710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 305/592] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:20.902794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:20.903814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 288/593] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:20.905935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.046616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94408585-d112-4940-88e4-03a0b1e552b3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.064571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfea69e-f2a7-4c1d-9d8b-8e877378e934 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.076485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.085479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.086440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.092211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.092211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.092211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:21.092211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.092211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.109896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 289/594] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:21.111160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.115876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.116753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.117032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.117305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.119097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:21.119413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.121020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.125231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.142549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 306/595] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Jun 24 14:24:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:21.143786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.247395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc702ad-0ffa-4d4c-85f8-ca342d3e417e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.254433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ad49e-dbc4-4b21-bd9b-d581db28114c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.288269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.289191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.289478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.289757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.290811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:21.291375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.292384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.292930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.293618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.294088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.294585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.305887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.307806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.310256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 290/596] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:21.311215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:21.328949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:21.329842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:21.330810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:21.330810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:21.330810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:21.330810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:21.332936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 307/597] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:21.333927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.363758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.395760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.424828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a948964-dbfa-43d0-b8f6-8727d97a644c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.453557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f32330-c560-45c8-8079-ee1b1d24f188 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.460891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.461866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.462297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.463435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.474847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.476499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.489920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.490994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.491368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.491737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/bb628c1d9c854cbca433d36d143374c7/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:21.496542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:21.497585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:21.498683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:21.498683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:21.498683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:21.498683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:21.500547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 291/598] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:21.501540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.511343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=bb628c1d9c854cbca433d36d143374c7, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.512976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.609269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 308/599] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:21 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/bb628c1d9c854cbca433d36d143374c7/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:21.611732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.618417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.632686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff63a9f-3dae-4192-8b4f-251cbb33de5f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.707273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.707949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.708226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.708498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.712605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:21.714645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.717813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.733787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 292/600] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:21.738866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.814407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:21.820184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923f44c-1909-423c-8f0e-d7f94f2c2865 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.858191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.859023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.859426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.859722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.860510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:21.860884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.862330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.878686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 309/601] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:21.880236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:21.887300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8a8d8-fc36-4961-8ca6-43d541b6138e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:21.934210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:21.936385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:21.936670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:21.936942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:21.953472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:21.956029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:21.983001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:21.983927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:21.984823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:21.986852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 293/602] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:21.993165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.027578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.035768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a586d-adfa-4853-9f87-dd033ea88b2b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.081681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.083562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.083847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.084125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.092278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:22.093786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d27925b6-e0b6-4163-a1ae-f5680bce2501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:22.109594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:22.110547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:22.111463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:22.115412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 310/603] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:22.116746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0d4a-eb51-4764-a6cc-dd96e3b14ce6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.117179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0d4a-eb51-4764-a6cc-dd96e3b14ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.117284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe0d4a-eb51-4764-a6cc-dd96e3b14ce6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.122558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 311/604] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:22.122558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0d4a-eb51-4764-a6cc-dd96e3b14ce6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.140915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.143739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb228e-7dc1-466e-a86e-c414eba1ab55 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.191162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.192195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.192692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.193177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.194369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:22.195449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:22.198316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbea3a3-054a-4acf-8600-dda1247a4d56 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:22.216355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 294/605] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:22.217974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.218434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.219459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.233166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe0d4a-eb51-4764-a6cc-dd96e3b14ce6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.241448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:22.243510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:22.273319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.274089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.274438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.274814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.276435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:22.276864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:22.278425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:22.296115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 312/606] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:22.297164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.342678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 295/607] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:22.342678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.360856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.404813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.429005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338f17a-db8b-4b81-b940-eec7c2945809 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.465741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.467331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.467842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.468495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/c42ba8788bf2431a96b2087b287d7426/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.471204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc53a38-ce2a-46cc-8ee0-e756e6c8bdd0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.485265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=c42ba8788bf2431a96b2087b287d7426, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:22.487108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:22.508011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.508901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.509184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.510223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/d08dcd615d87450d9ccdc58fe89f4014/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.527758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=d08dcd615d87450d9ccdc58fe89f4014, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:22.529303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd502232-018b-44f7-9717-90b738fc02aa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:22.556967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 313/608] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:22 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/c42ba8788bf2431a96b2087b287d7426/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:22.557907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.627787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 296/609] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:22 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/d08dcd615d87450d9ccdc58fe89f4014/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:22.629843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b34c1-98f9-4328-9a05-9946d37c2361 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.630214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b34c1-98f9-4328-9a05-9946d37c2361 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.630915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b34c1-98f9-4328-9a05-9946d37c2361 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.637451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 297/610] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:22.641389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.641678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.641959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.664730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:22.667285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:22.730218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c86c0-a43d-4e19-aece-a53695642bd3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.758665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.762798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:22.776473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.777501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.777861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.778209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.781956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 298/611] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:22.792432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:22.800780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:22.802104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:22.802992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 314/612] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:22.810269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.812595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:22.938266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d09383-4ec2-4391-b0b8-f752ca24d4e7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.939560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a692004-6a95-44fa-b7f9-f00d13622211 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:22.992795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.993593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.994417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.994714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.996470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:22.997141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:22.997423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:22.997701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:22.998457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:23.000350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.001758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.026243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 299/613] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:23.027456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.028829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:23.040269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:23.041269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:23.042129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:23.051012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 315/614] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:23.055739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.114600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:23.148034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:23.211089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bb6fe-ce5f-435a-a09b-f5d332b3a599 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.233148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760bc80-9a20-44c9-8457-24750d41966a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.271730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.272778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.273203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.273679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.287601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.289260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:23.308782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:23.309883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:23.310967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:23.310967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:23.310967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:23.310967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:23.312804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 300/615] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:23.317215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.317215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.317215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.317215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.317696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:23.318005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.318845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.319417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.338560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 316/616] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:23.343828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.425965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:23.430444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a173d0-913b-4819-905d-bb8db94ba01c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.473047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.474059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.474549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.474997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.476121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:23.476602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.478785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.503983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 301/617] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:23.508839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.565270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6ce49-6258-42d9-9a9c-658a5693a2f6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.584264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:23.630850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.631643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.631924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.632193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.649104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.652644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.679140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c846e2cd-32bd-47f5-9afb-e8e066cfedd5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:23.686096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:23.687004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:23.687883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:23.687883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:23.687883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:23.687883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:23.688475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 317/618] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:23.694981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.743277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.744562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.744828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.745102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.755610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.757097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:23.785162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:23.786155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:23.787155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:23.787693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 302/619] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:23.793776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.836089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:23.868344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6678419-937c-4136-9641-fe5ed78607b0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:23.910867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:23.911581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:23.911856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:23.912129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:23.912842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:23.913146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:23.917980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:23.946292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 318/620] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:23.950910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:23.989802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.021393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f2d6-f7c3-4953-8958-acd45ce1dbe6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.058112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.058948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.059241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.059523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.060420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:24.060741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.062198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2536e9c-5ea3-482d-ba4a-43030b3b9806 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.084851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 303/621] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:24.085775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4485b80-dc34-47aa-a1e4-47f1929910b6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.086228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374334fa-a7ca-49a9-9ed6-2155ba5f3406 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.086549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374334fa-a7ca-49a9-9ed6-2155ba5f3406 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.087036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374334fa-a7ca-49a9-9ed6-2155ba5f3406 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.092678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 304/622] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:24.093467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374334fa-a7ca-49a9-9ed6-2155ba5f3406 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.127619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.128415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.128700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.128974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.137551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.139539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b5db2bbd-6722-4390-8850-fbe0397c36de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:24.158883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:24.160258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:24.161190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 319/623] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:24.162261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.162418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.167588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.177857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.183491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:24.184941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:24.204108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374334fa-a7ca-49a9-9ed6-2155ba5f3406 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.241746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.242724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.243270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.243764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.270537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 320/624] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:24.270537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.290855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.343108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.344996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.388520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1e5cb-615d-40e8-a134-e9936a3702fc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.433247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.434350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.434861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.435319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.436334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:24.436875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.438536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.444333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.460059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 321/625] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:24.461296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.475439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 305/626] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:24.476636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.544795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.584086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cf2d-cb8a-4b4d-845e-ee75c38f4e83 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.590440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188ceaf-5072-4179-912d-0132375c2370 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.619698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.620595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.620943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.621287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.631151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.632197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:24.632454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.632937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.633392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:24.640514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:24.641517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:24.642372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:24.646615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 322/627] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:24.647517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.649028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.651512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:24.672228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:24.673297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:24.674201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:24.674201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:24.674201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:24.674201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:24.676718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 306/628] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:24.677993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.712026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.751734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57f927-bba4-4f38-a088-240d46b4c6fb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.778572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.786704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.787808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.787808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.788094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.795169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da76dd-7104-495e-ade3-c586527e0113 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.800258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.802093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:24.820366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:24.821289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:24.822225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:24.822225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:24.822225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:24.822225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:24.826180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 323/629] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:24.826965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.837296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.839052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.839538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.840013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.841096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:24.841676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.843500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.854142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 307/630] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:24.855342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:24.935454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a4ac8-0b4c-435b-8f9d-3d64131f5d7a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.958291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:24.967454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf09870-10e5-48e0-ac82-2a01d9b4cd2c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:24.970007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:24.970706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:24.970988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:24.971260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:24.971973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:24.972279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:24.973658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:24.985257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 324/631] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:24.986048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.001198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.001863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.002145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.002420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.003170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:25.003477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.004841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c3637-f198-4edf-a554-a4f8357384a6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.017424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 308/632] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:25.018689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8b64f-ec2d-495f-b557-0e1667892cf4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.018959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8b64f-ec2d-495f-b557-0e1667892cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.019234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8b64f-ec2d-495f-b557-0e1667892cf4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.020448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 309/633] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:25.021780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93411cb-555f-462c-8664-d8736d931ee3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.021899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93411cb-555f-462c-8664-d8736d931ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.022103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93411cb-555f-462c-8664-d8736d931ee3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.023510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 310/634] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:25.024033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93411cb-555f-462c-8664-d8736d931ee3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.034324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.093075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a88da7-a6c7-480c-8cfe-cce029dd5c15 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.125568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93411cb-555f-462c-8664-d8736d931ee3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.128131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.128929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.129210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.129486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.140774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.142222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.158494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.159295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.159583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:25.159702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.160954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.161851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:25.161851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:25.161851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:25.161851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:25.162073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.162144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 325/635] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.162144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.171303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.172755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6e876060-d429-4075-bee8-b0cc14ed0d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:25.187541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.188499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:25.189391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 311/636] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.190011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d3a6-d868-4684-b9a4-1f4e361c4c2a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.190204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d3a6-d868-4684-b9a4-1f4e361c4c2a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.190409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d3a6-d868-4684-b9a4-1f4e361c4c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.207119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b471d3a6-d868-4684-b9a4-1f4e361c4c2a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:25.208313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b471d3a6-d868-4684-b9a4-1f4e361c4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:25.216396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.258036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.266671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1359c11-675d-4828-8455-cdd1f9836f25 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.275397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 312/637] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:25.276742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.277025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.277303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.293433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:25.294641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:25.302779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.303571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.303852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.304124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.311024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.312462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:25.326139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.327114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:25.328014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 326/638] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.328496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.360601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 313/639] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:25.361374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.432101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.436095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06a12f-f34f-47d4-8e58-02577cc100eb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.460343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf8ca9-3a82-4b89-8041-027f36d5fe0f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.468863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.469522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.469803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.470075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.470820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:25.471134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.472513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.484825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 327/640] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:25.485622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.499345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.499707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.500357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.500634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.500910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.501686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:25.501990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.503464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.514773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 314/641] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:25.519670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.587001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7817f104-dd8d-48a4-a5f7-507766baf3bd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.616011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18323df6-7765-4a62-8bd7-89bb4afc12f7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.622682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.623479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.623755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.624024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.636825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:25.644833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.646010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:25.647113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:25.647904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.648000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 328/642] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.648000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.654350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.655466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.656004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.656473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.657883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:25.658408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.660978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.675432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 315/643] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:25.676473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.733050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.762726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18d8fb-db9f-4161-bbb5-d3675bee5c6a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.778664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9b781-8cfc-4d90-b7af-a309570efd73 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.795249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.796035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.796311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.796579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.807261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:25.812659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.813440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.813718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.813991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] PATH_INFO: `/v3/users/glance` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:25.814881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.820961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 329/644] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.822416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.829107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.830575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:25.844991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:25.846155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 316/645] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:25.847497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:25.900923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:25.917051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d6272-d8d2-44a9-9f3b-8d5ad81dae98 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.945467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bc788-c599-4521-86f6-2145f91a8cda None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:25.949555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.950441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.950779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.951054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/52486f194961487ba574eebe49869a96/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.965357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=52486f194961487ba574eebe49869a96, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.966828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.977648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:25.979002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:25.979280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:25.979550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:25.980279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:25.980581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:25.981926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:25.989488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 317/646] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:25.990244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.083579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb08dc-f346-4b7a-a93f-3373d5d2fcaa None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.115394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.116031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.116315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.116595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.117405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:26.117715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.119672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.124653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.125340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 318/647] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:25 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:26.128042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.204597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 330/648] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:25 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/52486f194961487ba574eebe49869a96/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:26.213803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.215678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.242380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d015bb-7728-44e3-b2f3-2f03b318e713 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.283845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.284538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.284821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.285094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.285857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:26.286162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.287625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.297194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 319/649] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:26.298101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.358510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.388255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01463f7d-580c-479f-8a09-f4c75a8338ba None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.434262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.435213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.435496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.435772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.436620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:26.436931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.438366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.441279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5285006-3e92-4117-bfa5-27f3c4d1d513 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.458994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 331/650] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:26.460084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.486645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.488290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.488855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.489290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.503364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.505871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.521589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:26.527029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:26.527937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:26.528847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:26.528847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:26.528847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:26.528847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:26.531157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 320/651] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:26.532208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.586469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ab645-019b-4d2a-8873-c56ed9dd4335 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.625549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.626579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.626774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.627055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.634869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.640181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.641649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.645281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5b07a-144f-4001-a3cd-b7755c060849 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:26.661182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:26.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:26.663056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:26.663056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:26.663056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:26.663056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:26.667036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 332/652] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:26.668486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.685874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.686889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.687416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.687866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.763750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.763750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.775492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.793317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259162e-3910-4247-b5f3-6a5ecbcade3c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:26.840814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:26.841479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:26.841760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:26.842051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:26.842799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:26.843113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:26.844489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:26.856642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 333/653] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:26.857671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.871248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:26.885158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 321/654] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:26.888716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:26.983114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920de1e-e7a5-4719-9fa3-1d9584d8abe9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.006247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5b944-6be3-4c47-bc6e-f48d47c45a1a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.019779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.020341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.020698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.021053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.021841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:27.022224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.023837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.036968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 334/655] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:27.038006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.051867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.053033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.053528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.053863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.054202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.062626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.072261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:27.080301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:27.081220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 14:24:27.082151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:27.084688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 322/656] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:27.085661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.140974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.144133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562295-fc5e-4bc6-8f7e-d13c9023fad2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.180537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.181911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.182191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.182457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.191090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7fc88-828b-4767-8882-b8cc92393d76 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.194565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.196054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:27.214569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:27.215921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:27.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:27.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:27.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:27.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:27.220409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 335/657] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:27.221234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.231227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.232398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.232885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.233444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.241610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.244283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:27.260214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:27.261182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:24:27.262131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:27.264759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 323/658] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:27.265753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.289606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.333214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dc3a2-8781-42d6-aa60-223ca50beb80 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.382949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.384691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.385141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.385966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.387673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:27.388306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.389968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ccc5b-1754-44d2-80a0-8d4364ba6457 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.390104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.405895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.416198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 336/659] 10.222.0.23 () {56 vars in 1139 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:27.421890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.429964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.430903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.431351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.431776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.432657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:27.433072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.434645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.454759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 324/660] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:27.455780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.545485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf5c9-ab3a-4ad6-9500-f4ba33307d67 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.568587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bea0de-f3ef-4d87-8f13-e7fa650d4f66 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.576066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.591469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.592864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.592864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.593166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.594580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:27.594849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.596800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.605470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.606229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.606705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.607081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.618987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 337/661] 10.222.0.23 () {56 vars in 1137 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:27.619786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.659728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.688804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.690403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.755951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5079229-69e0-48f0-b7d3-ffff3e9c0897 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.798303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 325/662] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:27.799259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.806565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.807105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.807660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.808100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:27.817539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:27.820726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:27.842829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:27.843988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:27.845083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:27.847313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 338/663] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:27.848286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:27.894387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.900427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:27.937869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c35ede2-e30c-48ec-a0da-81d40fbb7b29 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.966154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d882f-bab8-4bd7-87c1-3e047de7edac None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:27.981249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:27.982062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:27.982344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:27.982638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.012387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.013524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.014062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.014676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.026707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.028716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:28.048886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:28.049822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:28.050749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:28.050749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:28.050749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:28.050749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:28.058945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 339/664] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:28.060071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.066394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.067921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.155912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.163053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.172434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf0ee1-36b6-4b36-ae96-a7c906322f39 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.193621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 326/665] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:28.194580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.220605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.221277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.227279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.227279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.227279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:28.227279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.227279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.247482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 340/666] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:28.247482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.319247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da193479-fcff-4a63-852e-4bd2fddbc5dc None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.361313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.361996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.362291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.362573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.363307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:28.363616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.365000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.377701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 327/667] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:28.378362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.386291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a1ed8-559a-47ef-a12f-9c37349c8e02 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.409342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.419083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.419875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.420156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.420434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.431746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.433280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-84e00111-322b-4b4c-880a-81d265b19a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:28.450647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:28.451693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:28.452625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:28.452625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:28.452625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:28.452625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:28.452625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 341/668] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:28.453453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ea2c-a477-4bea-953a-104992e1c1a4 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.453782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ea2c-a477-4bea-953a-104992e1c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.454060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71ea2c-a477-4bea-953a-104992e1c1a4 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 342/669] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdc3d0-28b1-47dd-a4ff-0aef8aa446ce None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdc3d0-28b1-47dd-a4ff-0aef8aa446ce None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdc3d0-28b1-47dd-a4ff-0aef8aa446ce None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 343/670] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:28.459562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdc3d0-28b1-47dd-a4ff-0aef8aa446ce None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.462856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.482579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4aaa7a-11a8-4673-9b5a-070f96fb0f7c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.516059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.516715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.516991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.517264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.518075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=bb628c1d9c854cbca433d36d143374c7) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:28.518386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.519835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.525498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 328/671] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/role_assignments?user.id=bb628c1d9c854cbca433d36d143374c7&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:28.526255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.568172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdc3d0-28b1-47dd-a4ff-0aef8aa446ce None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.608513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.609421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.609806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.610180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.618106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.620366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.632001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f08cd104-6c38-4a57-b940-b8b06bacde68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:28.635263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:28.636170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:28.637058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 344/672] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:28.638251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.638749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.645028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.645304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed114fc-3f53-4ee3-8523-f620bd099d34 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.656820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:28.658050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:28.685002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.685887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.686007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.686211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.687159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:28.687262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.688639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0020e-18aa-4643-8346-eaf0bbd16c4d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.705356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 329/673] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:28.706961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.707253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.707888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.726911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:28.728286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.728635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:28.738182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 345/674] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:28.739296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.810023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 330/675] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:28.811173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.862980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18589e21-c5dd-4771-862a-67040d9ef302 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.899308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.900029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.900392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.900670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.901455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:28.901762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.903239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.906427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.915641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 346/676] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:28.916734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:28.927436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd87274-565e-4851-9d20-8c3078f3858f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:28.970381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:28.971445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:28.971899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:28.972311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:28.975254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:28.980713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:28.982277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:28.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:28.997810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:28.998925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:29.005146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 331/677] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:29.006184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.025510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bd986-1fdc-4b24-92b9-f37748d370ac None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.064452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.065251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.065534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.065809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.079549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:29.087323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:29.088237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:29.089077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 347/678] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:29.090290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.119249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b3e2e-35f4-46fb-9605-a98688fe231f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.155355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.156148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.156431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.156808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.166967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:29.174556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:29.175477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:29.176580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 332/679] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:29.177278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.187435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.200734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a59b24-6c26-4866-943d-e901232b372d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.215681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.237056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.237751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.237995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.238265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.239101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=d08dcd615d87450d9ccdc58fe89f4014) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:29.239416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.240830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.247298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 348/680] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/role_assignments?user.id=d08dcd615d87450d9ccdc58fe89f4014&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:29.248109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.292635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1884c5-4a61-4937-ad72-d80c590e855a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.353335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.355101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.356029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.356546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.358591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:29.359222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.360933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.379247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 333/681] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:29.380636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.401122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc0ede-7f02-4094-acab-ae564b6d3fb9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.448335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.449304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.449585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.449859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.450805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:29.451469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.453153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.468855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 349/682] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:29.469155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.476132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.515416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e10f80-1899-48f7-8d90-0029c3661db9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.518006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.563058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.564394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.565317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.565780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.566948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:29.567405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.569579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.582495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 334/683] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:29.583630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.592525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058e29e-54eb-4757-8ba9-7f89fd39e8b5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.628473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.629195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.629572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.630308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.630978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=c42ba8788bf2431a96b2087b287d7426) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:29.631412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.632937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.646546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 350/684] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/role_assignments?user.id=c42ba8788bf2431a96b2087b287d7426&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:29.646546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.781990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7c097-918b-413c-b49b-2f93079368e0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.785226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50f2fb-878e-4faa-8be7-ede81ded4868 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:29.785820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.822745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.823790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.824202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.824552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.842800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.844420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.857521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:29.869918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:29.870785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:29.877943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:29.877943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:29.879696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:29.880568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:29.881467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:29.881467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:29.881467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:29.881467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:29.883784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 351/685] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:29.890175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:29.894026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:29.895755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:29.917478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:29.918425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:29.919331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:29.919331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:29.919331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:29.919331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:29.921787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 335/686] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:29.927623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.049018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360142ae-e9d2-4820-9d60-6a72dc239584 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.061444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6bb9c5-2539-456f-ae63-95e93b205143 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.121772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.122442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.122774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.123048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.123772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:30.124077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.125446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.128359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.129028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.129303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.129573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.130395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:30.130677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.132086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.146828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 352/687] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:30.155641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.157138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 336/688] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:30.162447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.198179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.212635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.277952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26080361-fd62-4dfb-83e1-ba266d8acaea None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.317499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523942f-a385-4559-b157-69c62924c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.318304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523942f-a385-4559-b157-69c62924c621 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.318639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523942f-a385-4559-b157-69c62924c621 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.318920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523942f-a385-4559-b157-69c62924c621 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.327641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523942f-a385-4559-b157-69c62924c621 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.329197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523942f-a385-4559-b157-69c62924c621 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7523942f-a385-4559-b157-69c62924c621 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:30.346029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:30.347046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:30.348337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 337/689] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:30.359102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523942f-a385-4559-b157-69c62924c621 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.415951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51855e-5c60-4d6c-9f63-0833f2c446d2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.477192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.478730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.480189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.480857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.496994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.500696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:30.526839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:30.527833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:30.528775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:30.531557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 353/690] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:30.540803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.542731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.547495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523942f-a385-4559-b157-69c62924c621 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.630976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.631787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.632076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.632466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.633333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:30.633648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.635891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.642876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.674237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 338/691] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:30.677067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fd9fc-d4ad-489f-850e-521f9cf56c44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.680496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.741186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.742260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.742759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.744177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.745474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:30.745994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.747687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d54cd-44dc-43ec-aa3b-257fea089a08 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:30.777367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 354/692] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:30.778948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d83df-7358-4fd1-b626-5bb12036411c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.779244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d83df-7358-4fd1-b626-5bb12036411c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.779526 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d83df-7358-4fd1-b626-5bb12036411c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.781122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 355/693] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:30.783422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d83df-7358-4fd1-b626-5bb12036411c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:30.832964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241eeb3-ba58-4aa8-b19c-6658dd2ba4e5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.868640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.869588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.870094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.870583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.897553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d83df-7358-4fd1-b626-5bb12036411c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:30.898987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.936244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:30.936858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:30.937145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:30.937422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:30.940145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:30.943376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:30.945241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950fcdf5-78df-420d-9af7-42ab3b4dd010 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.014771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.016244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.051874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 339/694] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:31.053729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.054254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.054782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.073331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:31.074872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:31.128416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 356/695] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:31.129288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.130780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.145432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 340/696] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:31.146625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.186923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.249114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec43523-bd41-4b03-a0ee-9c72f97d969f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.268823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca30d0-1023-4f47-a5ec-a345568212f5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.284571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.285769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.286257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.286746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.297731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.299515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.306503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.307334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.307617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.307895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.318825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.320279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:31.320869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:31.322394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:31.323301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:31.323301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:31.323301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.323301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:31.325550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 341/697] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:31.326716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:31.340678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:31.341594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 357/698] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:31.343076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.382185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.438306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.442923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65389380-467e-47b9-8efb-384c3efbcdb2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.456332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b871-6f74-4693-b4c2-7c74afe7bb44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.478564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.479519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.479894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.480273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.490848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.492634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.496596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.497444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.497907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.498332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.499234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:31.499691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.501322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:31.512498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:31.513444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:31.514345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:31.514345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:31.514345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:31.514345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:31.516679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 358/700] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:31.517799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.518602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 342/700] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:31.520055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.622042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.629524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ee908-74fe-4803-b416-5c313481d72e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.631320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f396a7a-fca5-4c8d-8911-64842e08fd27 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.667285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.667953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.668340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.668623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.668812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.669473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.669753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.669986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:31.670081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.670307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.670765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:31.671072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.671723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91aed0c-82c1-4d78-9a84-7d726c07271e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.672435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.689254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 343/701] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:31.689254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.689510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 359/702] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:31.689510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366541c-f109-43e5-9dce-c11af1965fef None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.689510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366541c-f109-43e5-9dce-c11af1965fef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.689510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366541c-f109-43e5-9dce-c11af1965fef None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.689754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 360/703] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:31.694456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366541c-f109-43e5-9dce-c11af1965fef None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.703916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.799125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f108c-d35b-44a9-a27e-12fd35624f1e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.805362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366541c-f109-43e5-9dce-c11af1965fef None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:31.836373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.837221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.837582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.837856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.843161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:31.843883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.844210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.844501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.844829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.845211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:31.846277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.846752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120b8a95-c291-494f-b862-6c7cfa1cfaa8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:31.861347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:31.862499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:31.863574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 344/704] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:31.864079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:31.879274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:31.933716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 361/705] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:24:31 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:31.935303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:31.935707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:31.936045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:31.954490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:31.955911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:31.984959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e2094-7e51-4207-8b7f-8ab1ec10aa78 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.023610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.029005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.029819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.030105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.030380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.048239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 362/706] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:32.048239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.048561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.048561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-adf8bdc6-47b9-471b-8e4f-87e347ec1d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:32.059600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:32.060806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:32.061912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.063259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 345/707] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:32.064676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2b88e-a7de-452d-a644-9d02ee9902d8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.065053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2b88e-a7de-452d-a644-9d02ee9902d8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.065394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b2b88e-a7de-452d-a644-9d02ee9902d8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.070585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 346/708] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:32.070585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2b88e-a7de-452d-a644-9d02ee9902d8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.151567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.175259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150310a6-29ee-46dd-8fe4-4a26f16b777b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.183892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b2b88e-a7de-452d-a644-9d02ee9902d8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.216462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.217075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.217346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.217615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.218396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.218738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.220175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.221588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.222591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.222873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.223145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.223790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.225143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.237040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 363/709] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.237040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9824b8af-bb75-48f1-8514-9786d75f42c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:32.241051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:32.242063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:32.243471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.244350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 347/710] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:32.244961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.245229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.245501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.263781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:32.265072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:32.291607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.341026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 348/711] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:32.342214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.362333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7e244-7d2a-433d-b336-8b16b23e41f6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.400685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.401353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.401636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.401918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.402731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.403046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.404470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.417509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 364/712] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.418439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.436549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.475027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9b2a5-b89d-4444-8a0d-6f2ba4c5573b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.511629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.512416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.512699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.512981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.513801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.514112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.515885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.520987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a29e03-e715-48d5-be83-23ba83acc4d8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.526565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 349/713] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.526565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.549675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.581206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.581920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.582629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.582973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.584023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] RBAC: Adding query filter params (name=network) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.584330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.586704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.599340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 365/714] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.600646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.656891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61383f4e-0e4a-4096-9503-d1744fc2aa57 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.691537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.698957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.699905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.700247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.700578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.701349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.702822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:32.716708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:32.717747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:32.718669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 350/715] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:32.719538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.743979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5db0-b514-4ea7-81ec-dca990a8310f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.784465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.785116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.785394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.785665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.786485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=52486f194961487ba574eebe49869a96) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.786822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.788240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.797048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 366/716] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/role_assignments?user.id=52486f194961487ba574eebe49869a96&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.798328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:32.867876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f247d-b52c-4014-aadb-2017def4b14b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:32.878036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:32.933547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:32.934225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:32.934504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:32.934805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:32.935608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:32.935912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:32.937297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:32.956147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 351/717] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:32.994274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60e069-0258-4370-9426-257d395a51a9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:33.036365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:33.037333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:33.037766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:33.038189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:33.039105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:33.039592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:33.041288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a26448-f9d9-478a-8916-6de67ddd30bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:33.046997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 367/718] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:33.052591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:33.135970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:33.184252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2606e0a-5e8d-4ba9-9653-77e53ff57576 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:33.220589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:33.221555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:33.222024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:33.222532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:33.223494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:33.224001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:33.225557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211620d6-2ca9-4b47-ac83-fd69797b65f7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:33.231228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 352/719] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:34.224785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa3db-90d3-4efa-b834-c512ee655f3b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:34.225336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa3db-90d3-4efa-b834-c512ee655f3b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:34.225336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa3db-90d3-4efa-b834-c512ee655f3b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:34.226948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 368/720] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:34.232743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:34.233191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:34.233283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:34.260352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:34.261548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:34.333704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 353/721] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:24:34.370348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa3db-90d3-4efa-b834-c512ee655f3b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:34.503365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa3db-90d3-4efa-b834-c512ee655f3b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:34.512256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:34.553583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:34.554381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:34.554691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:34.554962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:34.568769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:34.577146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:34.578087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:34.578957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:34.582619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 369/722] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:34.586939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:34.657303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:34.718867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523768ba-d841-4436-9135-6b2a4b6dfda1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:34.767324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:34.767913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:34.768190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:34.768466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:34.769209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:34.769516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:34.770904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:34.781782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 354/723] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:34.787231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:34.901113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2604742-052d-4447-bfd7-e76ac73f9995 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:34.945544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:34.946895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:34.947176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:34.947456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:34.958656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:34.960108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f6019bd0-eef2-446a-8a79-ce0a1311c8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:34.988185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:34.989226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:34.990135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:34.990135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:34.990135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:34.990135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:34.990135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 370/724] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:34.995440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.028870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf8a3-afad-41fb-ade9-d3520a2a0825 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.029264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf8a3-afad-41fb-ade9-d3520a2a0825 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.029264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaf8a3-afad-41fb-ade9-d3520a2a0825 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.030863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 371/725] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:35.036015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.036300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.036575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.053509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:35.054707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:35.066748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.130628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 372/726] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 14:24:35.147489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.193292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d5b4-7f0f-476f-81aa-054d17d3181b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.242183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.245298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f535a-0bfc-4394-8f0f-fbd88bd5e50c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.280361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.281076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.281354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.281628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.287789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.288465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.288751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.289030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.290346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:35.290684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.292078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.293016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:35.300767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:35.301684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:35.302565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:35.306638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 373/727] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:35.308292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.322995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 355/728] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:35.328357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.329063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.427012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a9e6-84b8-40d5-8636-15325e98486c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.474081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.474775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.475051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.475318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.476065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:35.476370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.477760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.502976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 374/729] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:35.504480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.550881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77c498-440c-4fe3-932d-0847f2a90590 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.616384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.639246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.640842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.640842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.640842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.642620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbfa2f-948a-4ec4-9257-6a90e603e763 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.659406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.660879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.680444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.681288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.681564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.681836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.694276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:35.695569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:35.697073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:35.698731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:35.699299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.699383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 356/730] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:35.707517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0554d9ba-7e7d-4e3e-9f49-d8264128b285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:35.721781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:35.724177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:35.726288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:35.726288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:35.726288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:35.726288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:35.726820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 375/731] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:35.728144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51271-966e-4a24-99b4-3b4c45d6f691 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.728614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51271-966e-4a24-99b4-3b4c45d6f691 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.728791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd51271-966e-4a24-99b4-3b4c45d6f691 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.734557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 376/732] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:35.734557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51271-966e-4a24-99b4-3b4c45d6f691 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.759689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.879826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd51271-966e-4a24-99b4-3b4c45d6f691 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.880193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdaee-9d14-49f3-a318-abdb6887b3d1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:35.905997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:35.923794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.924607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.925030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.925635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.925748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:35.926488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.926623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:35.926930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.927040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.927324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.928402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:35.928775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:35.928899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192cdcbd-84e1-4416-abc2-ce33e059280b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.931223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:35.949176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 377/733] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:35.951456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9bf03-aa10-4268-873e-792bf34ce891 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:35.951746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9bf03-aa10-4268-873e-792bf34ce891 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:35.952017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9bf03-aa10-4268-873e-792bf34ce891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:35.958650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 357/734] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:35.958650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:35.975680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-17f9bf03-aa10-4268-873e-792bf34ce891 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:35.976873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-17f9bf03-aa10-4268-873e-792bf34ce891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:36.045377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 378/735] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:36.046487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c776b-7a83-4d16-bd4b-826df0ae843f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.046809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c776b-7a83-4d16-bd4b-826df0ae843f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.047093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c776b-7a83-4d16-bd4b-826df0ae843f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.048443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 379/736] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:36.049043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c776b-7a83-4d16-bd4b-826df0ae843f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.061089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.070327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39d775-8757-4100-908e-f2c260faf5bb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.104647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.105490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.105773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.106051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.112991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.114434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5ff51f6d-d092-482f-bc56-709cb8ec4741 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:36.129436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:36.130414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:36.131396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 358/737] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:36.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035322d-a4c5-4d56-aee2-9ee615b8cec2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.132999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035322d-a4c5-4d56-aee2-9ee615b8cec2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.133303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035322d-a4c5-4d56-aee2-9ee615b8cec2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.134734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 359/738] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:36.135848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.136136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.136407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.152517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:36.153711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:36.160255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c776b-7a83-4d16-bd4b-826df0ae843f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.171406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.194401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.195077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.195357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.195630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.220144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 360/739] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:36.221015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.283233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.284694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.303525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.321674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b5fd-89e1-4dd8-b255-8210e68d51d2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.362150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.363133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.363420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.363699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.364423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.365970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b5a35-156d-414a-ba0c-782d41bbe59c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.388250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 380/740] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 14:24:36.389016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.400429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.464558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 361/741] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:24:36 2022] POST /identity/v3/services => generated 240 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:36.466084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.466366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.466945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.486605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db57a5-b725-4f33-9f1c-3616a5e0cce4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.502308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:36.503581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:36.543332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.544022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.544302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.544573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.545431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:36.545799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.547343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.573285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 381/742] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:36.573285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.602114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 362/743] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:36 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 14:24:36.602114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.641722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.683868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95dc64-780f-4535-95f5-714ba12b3451 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.694125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.721845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.722666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.722943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.723212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.731574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0668cd-f876-4292-a42d-1a2ffe261976 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.736367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:36.744944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:36.745941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:36.746840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 382/744] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:36.748273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.774494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.775411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.775660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.775934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.786772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.788593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:36.806958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:36.807896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:36.808789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:36.808789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:36.808789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:36.808789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:36.812634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 363/745] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:36.813612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.860962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cdd1c-f512-4bd0-bbe2-ece4e6c002a6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.884855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.897390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.898320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.898635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.899650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.910106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.911596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:36.918908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb8f87-950c-49e3-a451-dc90fa890de7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:36.931071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:36.932070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:36.932972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:36.932972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:36.932972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:36.932972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:36.932972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 383/746] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:36.933674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:36.948072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:36.961062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:36.962393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:36.962794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:36.963080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:36.963874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:36.965265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.034321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b97959-cb68-4989-91a7-d48bb5b33831 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.043584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 364/747] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:24:36 2022] POST /identity/v3/services => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:37.051793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.091114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.092244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.092675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.093192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.094210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.094792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.096489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.107888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 384/748] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.109279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.152099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.207154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908b5ad-8bcd-4d66-bbff-6884edf8524f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.239900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43030afe-cce9-43a0-b830-8990a46225a3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.244411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.244821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.245104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.245382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.246106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.246414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.247837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.260907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 365/749] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.261669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.285152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.285775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.286061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.286336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.287093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.287400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.288787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.309297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 385/750] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.310459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.324428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.393037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b014cb-852f-4ae2-8261-1bfca821fffb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.438796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.440029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.441654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.442736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.447796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.456095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.457851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.458788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0950c52-3401-45b8-9a39-8d7b9ff3b69e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:37.477851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:37.478867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:37.479850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:37.479850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:37.479850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:37.479850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:37.482231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 366/751] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:37.483402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.499408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.500547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.501027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.501509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.509063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.510791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:37.525670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:37.526601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:37.527499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:37.530032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 386/752] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:37.531192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.601061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.607670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da6252-b16e-4ca7-95ba-7f571151c8f3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.645602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.646397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.646718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.646988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.656886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.657796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872830c4-f951-4e44-b31b-8e6e849feab0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.658330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:37.676020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:37.676984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:37.677892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:37.682292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 367/753] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:37.683328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.702435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.703841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.704568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.705023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.706127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.706755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.708148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.725420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 387/754] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.726531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.747088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.801513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63121dcd-be2f-4173-9c02-766712753ed9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.838195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0397ac-4fc4-40a2-8f34-a4bf630c9ee4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.839008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.839943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.840470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.840959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.841936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.842443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.844183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.857806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 368/755] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.859067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.877400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:37.878064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:37.878341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:37.878655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:37.879436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:37.880005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:37.880117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:37.882248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:37.894619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 388/756] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:37.899787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:37.967571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669f010-d32c-426f-aec2-fe332ddfc4c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:37.980615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:38.003793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.004957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.005442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.005917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.006699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9445ee-0fca-4083-8757-24eb7095a809 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:38.014009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:38.015734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:38.031546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:38.032506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:38.033474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:38.036212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 369/757] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:38.037252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:38.047112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.048035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.048147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.048405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.049230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=8f0210144ae443e8a6d1b9e0fe196984) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:38.049538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:38.051365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:38.058634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 389/758] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:37 2022] GET /identity/v3/role_assignments?user.id=8f0210144ae443e8a6d1b9e0fe196984&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:38.059475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:38.136284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:38.161590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd499-6ff0-462b-85ea-a704a28a7e68 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:38.205102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.206243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.206706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.207087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.209651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=4bfc33870df145a4ba481afd9cdab92b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:38.212267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:38.305652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc63f96-f29f-47bc-b07d-722cafa271b8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:38.322159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 370/759] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:38 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:38.352511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.353447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.353860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.354255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.355320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:38.355781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:38.357441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:38.382922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:38.389507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 390/760] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:38.395295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:38.542982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe211-aa2e-4caa-b431-503e91e448e5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:38.579687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.580715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.581194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.581667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.586503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:38.654973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:38.656437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06993cac-5302-44e8-8daa-2429dcfbcb8e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:38.785132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 371/761] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:24:38.791602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:38.905300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b088d-9116-4f2d-ab10-491bc8f553f7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.905489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b088d-9116-4f2d-ab10-491bc8f553f7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.905770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b088d-9116-4f2d-ab10-491bc8f553f7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.907411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 372/762] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:38.912389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.912674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.912946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:38.936222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:38.937412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:38.943179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b057f-d8fe-4810-95f4-3ffb3a8c6f78 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:38.986981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:38.988134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:38.988420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:38.988692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.000076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.003624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.005075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:39.025710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:39.026719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:39.027624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:39.027624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:39.027624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:39.027624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:39.027624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 391/763] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:39.033719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.084719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 373/764] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:39.088873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.109896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.163103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae92b0-c0a7-42d0-be14-8756d6709f28 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.219258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f9428-1a8c-4943-a5c5-aff9c2da9fde None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.231227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.232216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.232715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.233180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.234118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:39.234609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.236116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.254873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.255866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.256241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.256792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] PATH_INFO: `/v3/services/network` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.257574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.259724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.264468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 392/765] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:39.271908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:39.282458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:39.283590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:39.284422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.293105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 374/766] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:39.296283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.332109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.404909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ccabd-1a15-45b4-acad-5dcd89286d3c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.419105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed280cf-c598-4be6-b4d8-3f550988a6c1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.428999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.439791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.440453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.440731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.441131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.441840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:39.448764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.448764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48c1f1-b8f5-4c0e-af15-e1b6b30ac1c4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.449104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 375/767] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:39.450345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f32f3-0098-46ce-aaa2-a19d4ad89388 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.450660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f32f3-0098-46ce-aaa2-a19d4ad89388 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.450932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f32f3-0098-46ce-aaa2-a19d4ad89388 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.452262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 376/768] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:39.457354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f32f3-0098-46ce-aaa2-a19d4ad89388 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.466480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.467460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.467744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.468015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.476081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.477794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3d5a88ce-fa86-4505-a67b-6a9bd26be340 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:39.493540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:39.494472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:39.495402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 393/769] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:39.496567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.496855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.497127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.515477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:39.516930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:39.568006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.576514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f32f3-0098-46ce-aaa2-a19d4ad89388 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.597779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 394/770] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:39.598818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.630599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.631320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.631600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.631871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.632625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:39.632988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.634465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.646923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 377/771] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:39.647752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.702841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.707038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd359ab-4d2d-402a-ba7d-e7e648cb2c6a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.743802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.745091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.745376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.745656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.746921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:39.747146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.749276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.759354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057fa0-582f-4513-8442-242b38e97557 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.769628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 395/772] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:39.770717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.802203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.803820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.803820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.803978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.804644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.806307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:39.819136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:39.820084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:39.821051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:39.825498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 378/773] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:39.826537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.857233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:39.889897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66152c1-bbcd-4854-923e-eabd454c633b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.925790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:39.926448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:39.926753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:39.927025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:39.927808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] RBAC: Adding query filter params (interface=public, service_id=de7e0c905df143fa82a9efbc5f5b6b51, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:39.928118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:39.929535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:39.942084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 396/774] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/endpoints?service_id=de7e0c905df143fa82a9efbc5f5b6b51&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:39.942976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:39.957834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fcab6-fb4a-4272-8712-a751676f8900 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:39.969354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.004109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.004965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.005245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.005517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.006339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=52486f194961487ba574eebe49869a96) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:40.006682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.008771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.018817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 379/775] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/role_assignments?user.id=52486f194961487ba574eebe49869a96&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:40.019733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.067303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fd288-ae32-4f81-8df4-dc009081a4f8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.112824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.113878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.114378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.114879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.115802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:40.116315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.118614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.125580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 397/776] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:40.134953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.220329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.249078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697599e-85e0-485d-8f0d-3875e16a9aef None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.294146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.310125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5070d-185b-4133-bc58-f3835c94225a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.317245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.317967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.318255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.318601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.319238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.320716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.335922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 380/777] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/services => generated 772 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:40.364110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.365152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.365626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.366122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.368077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:40.368607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.370191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.383060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 398/778] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:40.389248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.506628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983013ba-2052-4b3b-9ba4-840b4515268e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.550608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.551688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.552186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.552676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.553689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] RBAC: Adding query filter params (interface=public, service_id=177dd3490eb042219cc05ea994f77e94, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:40.554220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.555925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954962a-c1e9-4946-933e-cc9690089183 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.569757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 381/779] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/endpoints?service_id=177dd3490eb042219cc05ea994f77e94&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:40.576045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.615598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f672f-f98e-478c-a5a7-9d042e497edd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.616205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f672f-f98e-478c-a5a7-9d042e497edd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.617570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f672f-f98e-478c-a5a7-9d042e497edd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.619745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 382/780] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:40.629333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ff75-12c4-40cd-9a65-5157fa642b24 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.630065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ff75-12c4-40cd-9a65-5157fa642b24 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.630557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ff75-12c4-40cd-9a65-5157fa642b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.656036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d514ff75-12c4-40cd-9a65-5157fa642b24 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:40.656790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d514ff75-12c4-40cd-9a65-5157fa642b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:40.670595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.733656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f942f5-5c8d-48ae-9e5a-be167a93d934 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.761892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 383/781] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:40.763176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3baea1-f960-4525-baac-dd812d8f7781 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.763460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3baea1-f960-4525-baac-dd812d8f7781 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.763735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3baea1-f960-4525-baac-dd812d8f7781 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.764982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 384/782] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:40.780174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.780835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.781111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.782738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.783022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.783294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.785613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.789094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:40.791259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:40.797286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 399/783] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:40.798102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.802405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:40.803730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:40.816650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.886664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 385/784] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:40.904220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:40.911095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60281e42-2e45-4659-aa81-7a543fe7b19d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:40.951471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:40.952277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:40.952555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:40.952823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:40.965651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:40.973586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:40.974561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:40.975431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 400/785] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:40.995507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:40.996239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.031866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a24a85-77ff-47df-8076-285550f9cf58 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.071100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.072077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.072362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.072640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.083420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:41.092041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:41.093231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:41.094271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:41.098003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 386/786] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:41.099213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.121016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:41.146961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb13e4-f782-4402-bb21-57d2b380aa5a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.189291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.190349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.190840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.191296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.192233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:41.192801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.194370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.204719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2606d-47c8-4b4d-8e1b-5fa956afc0c5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.209543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 401/787] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:41.215928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.241075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.241748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.242032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.242986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.243999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:41.244309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.246842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.247278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:41.255501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 387/788] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:41.261802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.376899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39971fee-2b86-46cc-a8ac-ae33b192d61a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.415840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.416894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.417602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.418025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.428994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.431372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:41.450008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:41.450992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:41.451906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:41.451906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:41.451906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:41.451906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:41.458374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 388/789] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:41.459857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.496126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669925-0361-41be-9429-2c5bf282f4fa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.510912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:41.533947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.534418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.534778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.535242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.543089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:41.547289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.548800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:41.575218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:41.578566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:41.579500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:41.579500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:41.579500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:41.579500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:41.579500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 402/790] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:41.583593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.587713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb4381-66e6-4926-afca-c364f9a24b36 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.629655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.630691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.631176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.631648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.632589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:41.633631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.635846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.654128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 389/791] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:41.655835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.769987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527a9f8-1d20-4aa4-a624-35ca7fe806a9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.808193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.809339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.809823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:41.810298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:41.819078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:41.822137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:41.837923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:41.838954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:24:41.839914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:41.846886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:41.847172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 390/792] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:41.849463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:41.948328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34247487-5605-473f-959a-ee2c538d0977 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.960588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ea036-af7d-4154-9110-d4b19c7ce9ea None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:41.998123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:41.999198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:41.999682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.000156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.001181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:42.001716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.003392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.026699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 391/793] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:42.028503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.029123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.029255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.029453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.029726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.030455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:42.030785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.032145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.060863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 403/794] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:42.066142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.066948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.137587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fa0da-6edb-49a2-9948-b5085041fad9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.148924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.175409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.176933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.177417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.177909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.256018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.258566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.282735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59615e74-a193-4f63-955f-d9e8084fc625 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.318875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.371988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.372830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.373118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.373397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.395497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.396972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.413864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 392/795] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 14:24:42.416144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:42.432327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:42.433328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:42.434270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:42.442957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 404/796] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:42.448478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.533405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c698b-f800-4c7f-8066-61c54cd3cb72 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.570965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.572290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.572824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.573310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.581052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abc3a2-baed-4a7b-a00d-0355c092fe4d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.585345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.587146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.590499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.591368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-9fdaf18f-2764-4959-b7f0-c06c90bcfbb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:42.609691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:42.610664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:42.611566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:42.611566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:42.611566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:42.611566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:42.617731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 393/797] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:42.617731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239757-e186-4899-99e8-94951c36b6f2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.617731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239757-e186-4899-99e8-94951c36b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.617731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239757-e186-4899-99e8-94951c36b6f2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.617731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 394/798] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:42.619236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.619851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.620072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.620343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.621177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:42.621612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.622938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb1653-a4aa-49a3-bed8-8ef1444e8239 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.623580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4239757-e186-4899-99e8-94951c36b6f2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.641387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 405/799] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:42.646920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.647226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.647509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.666231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:42.666857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:42.738298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4239757-e186-4899-99e8-94951c36b6f2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.739750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 406/800] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:42.740785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.777216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.778355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.778921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.779396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.780320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:42.780823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.782417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.801159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 395/801] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:42.802450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.841646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.847537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:42.848146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc8510-dc53-470b-8e77-e64e47666456 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.885316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.886182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.886462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.886762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.897213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:42.899312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:42.912532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1921c-8ea3-4be3-969a-77876304a3e1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:42.916686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:42.917581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:42.918483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:42.918483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:42.918483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:42.918483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:42.922440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 407/802] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:42.923324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:42.950497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:42.951319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.951614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.951888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.963531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7acf4906-bd68-4990-a983-1bb6a7926135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:42.971112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:42.972085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:42.972944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 396/803] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:42.973670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16803-4d96-4656-9dbd-0794b014ff10 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:42.974235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16803-4d96-4656-9dbd-0794b014ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:42.974235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c16803-4d96-4656-9dbd-0794b014ff10 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:42.978732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 397/804] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:42.978732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16803-4d96-4656-9dbd-0794b014ff10 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.038675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0943952-0eba-4b5d-afac-bc6e6eaa8a90 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.081036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.081751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.082030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.082311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.082542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.087434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c16803-4d96-4656-9dbd-0794b014ff10 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.090237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.091161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:43.106885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:43.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 408/805] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:43.118103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.124998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.129844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.130535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.130824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.131098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.131818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.132127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.133718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc28324e-51be-45d1-becb-43f87c08cbd5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.145886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 398/806] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.147095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.147388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.147662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.164823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:43.166289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:43.229085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689a923-a12f-4a5a-906e-5abcbd37ca28 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.237755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 399/807] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:43.238618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.275062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.276002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.276442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.276718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.277471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.277779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.279181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.287377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 409/808] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.288174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.341190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.353025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805a29d-cfbc-44b7-a81f-bd70335f0385 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.365248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.391162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.391827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.392113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.392390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.393174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.393486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.395043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.396027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66ba0e-924d-4b51-ac10-89dc7d21bf2b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.406400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 400/809] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.407219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.428296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.429349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.429449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.430200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.437371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.438845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:43.453415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:43.454461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 410/810] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:43.455459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.511326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b29a3-75c4-4928-aec1-478eabb29da2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.543515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.544309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.544589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.544864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] PATH_INFO: `/v3/services/network` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.545511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.546911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.552714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a747ce-2b52-40fa-9f72-4e4510fcbe42 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-34eb50e2-915b-465b-a1f7-2e4837ae46ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:43.559576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:43.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:43.561383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 401/811] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:43.562205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803c8bb-2296-4904-9822-6e69d8f4d880 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.562205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803c8bb-2296-4904-9822-6e69d8f4d880 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.562441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803c8bb-2296-4904-9822-6e69d8f4d880 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.563724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 402/812] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:43.564409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803c8bb-2296-4904-9822-6e69d8f4d880 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.579982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.585801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.595108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.595108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.595108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.595108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.603375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.604809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:43.622732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:43.630258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 411/813] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:43.631193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.670799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803c8bb-2296-4904-9822-6e69d8f4d880 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.704467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.705114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.705393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.705664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.706476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.706817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.708232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.714473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 403/814] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.715261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.728119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d366fb-c6c7-400f-9839-43c413dd71be None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.770172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.770972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.771293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.771573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.772440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.772758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.775160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61febde-3fb3-4c50-a0d9-9f5252baf6d2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.796555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 412/815] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.798125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.798420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.798736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.801742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.818657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:43.819919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:43.826074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b309d75-d572-460d-be65-cf7a3cba7013 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:43.862253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:43.864024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:43.864699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:43.864986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:43.865262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:43.865971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] RBAC: Adding query filter params (name=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:43.866279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:43.868302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:43.875297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 404/816] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:43.876222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.904801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 413/817] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:43.905900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:43.992518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2dcf-80ae-4761-b678-fdec04743449 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.019326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83cfe2-ac1e-4a91-af66-daeb32405460 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.028974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.029645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.029961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.030231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.030993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:44.031305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.032684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.043505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.051634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 405/818] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:44.051634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.061205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.062255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.062667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.063010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/8f0210144ae443e8a6d1b9e0fe196984/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.078678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=8f0210144ae443e8a6d1b9e0fe196984, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.080251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.150680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b470e2c-ebfb-4813-8bb1-7b9957d39101 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.166796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 414/819] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:43 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/8f0210144ae443e8a6d1b9e0fe196984/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:44.167848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.196394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.197957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.198452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.198950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.199889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] RBAC: Adding query filter params (type=network) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:44.201004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.202989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.203713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.213291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 406/820] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:44.214629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.292993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.327712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e951ee3-1b43-4b48-9cbb-dea325de1a2d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.340621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae458db-12f1-40bd-9c12-04c208fb3c9b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.376463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.376942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.377222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.377794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.378905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.380541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.387506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.388306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.388589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.388863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:44.397457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:44.402020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:44.402925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 415/821] 10.222.0.23 () {56 vars in 1129 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:44.403780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.406311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.407863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:44.424404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:44.425656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:44.426626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 407/822] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:44.427231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.508469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.524381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057aae90-ee51-448b-aaa3-078fe5fa5435 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.564676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3ab12-967c-405f-a0ea-f4f7b576d5ac None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.571001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.571958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.572450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.573448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.574387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.576833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.591860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.602696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.603359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.603639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.603913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.604613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:44.604917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.606274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.610838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 408/823] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:44.611420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.681299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 416/824] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:24:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:44.683593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.746807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621537c-c1ba-45c7-9890-5ffc94e0db6b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.821695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9bcb9-ddda-42af-a7a6-1c4d545a4fa8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:44.831254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.831951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.832327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.832661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.833610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:44.833927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.835410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.861562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:44.862245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:44.862546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:44.862842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:44.863564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:44.863876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:44.865273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:44.873094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 409/826] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:44.877276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 417/826] 10.222.0.23 () {56 vars in 1124 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:44.878208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.878495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:44.881451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.988590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:44.988762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41352f6-0af2-479c-92a1-c3f7731cf55b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.025176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.029560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.029560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.029560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.029560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.029560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.057711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608302c-d6ec-4420-aff1-4dbce5f96d93 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.096290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.097152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.097449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.097728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.111118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.112628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:45.131705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:45.132711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:45.134357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:45.134357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:45.134357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:45.134357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:45.134357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 410/827] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:45.142670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:45.143021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 418/828] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:24:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:24:45.257390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:45.262895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d235c-9394-480f-8532-d392bd8171fb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.308195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.309210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.309672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.310123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.311055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:45.311542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.313101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.326204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 411/829] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:45.331679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:45.397648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:45.444471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f220c2-7a74-4263-8a0a-ef2b5615cfa9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.480112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.480923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.481204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.481479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.488659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.490144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:45.505754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:45.506792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:45.508296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 419/830] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:45.515486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:45.621447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af011d53-1731-43cf-92ba-09553687cc07 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.655929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.656591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.656876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.657154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.657938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:45.658247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.659728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60140e71-ecb9-449b-a383-9053b7b25e21 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.672645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 412/831] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:45.678752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:45.784473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf56ffd-9337-434c-8917-29825c8cb6b7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:45.804219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:45.821450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:45.822336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.822712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.822987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/52486f194961487ba574eebe49869a96/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.824938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=52486f194961487ba574eebe49869a96, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:45.830052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:45.930589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d1aa-3092-4a54-a7eb-ded6a44f7e19 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.930917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d1aa-3092-4a54-a7eb-ded6a44f7e19 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.931164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6d1aa-3092-4a54-a7eb-ded6a44f7e19 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.932722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 413/832] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:45.937505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:45.937796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:45.938078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:45.952460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 420/833] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:45 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/52486f194961487ba574eebe49869a96/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:45.958332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:45.959603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:45.971982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:46.037936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 414/834] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:46.063093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.170598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae9c02-5bdd-4efa-a500-7da33fe6d657 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:46.206131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:46.206984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.207270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.207547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.208204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:46.209696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b32bcaed-62b2-47cb-9c50-b72fc764852d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:46.223086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:46.224054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:46.224925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 421/835] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:46.230477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.334728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598be6aa-2226-4a3c-997a-9c76ab5bc56d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:46.363157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f024919-779e-4600-8d0b-f02b1d7b0ec9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.363589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f024919-779e-4600-8d0b-f02b1d7b0ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.363814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f024919-779e-4600-8d0b-f02b1d7b0ec9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.365107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 422/836] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:46.371794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:46.372476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.372758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.373037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.373754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:46.374063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:46.374473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.375012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.375473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:46.375673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.380877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 415/837] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:46.386575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.396245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:46.397727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:46.397836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:46.446832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:46.492938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 423/838] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:46.509439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9be993-d014-45aa-b9ef-06852c2a7cc3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:46.515569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.545823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:46.546864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.547244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.547671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.548470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:46.548874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:46.550599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:46.556296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 416/839] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:46.630489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca984782-7ec7-4618-9b8e-ecc8fbf33e3b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:46.668046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:46.669143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.669569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.669983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.683288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:46.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:46.692635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:24:46.693672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:46.696179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 424/840] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:46.702098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.784542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:46.815672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b863-2df8-490d-a386-29022c97b2b1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:46.852819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:46.853482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:46.853763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:46.854036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:46.854806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:46.855116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:46.856479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:46.864928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 417/841] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:46.870386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:46.906816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:46.978133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0adaf0f-a92b-4d64-9900-d2617ec2ab33 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.015451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.016258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.016533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.016802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.027528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.029067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:47.047126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:47.048045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:47.048952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:47.048952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:47.048952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:47.048952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:47.048952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 425/842] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:47.054062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.157614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb7d26-69fa-4ed7-a669-880574cc5ef6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.194054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.194739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.195647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.195921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.196636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:47.196940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.198319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8dd4b-f79f-4e0a-ba00-df3e876a816a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.199804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:47.328725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 418/843] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:47.334110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.392276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676042f0-29e3-4403-ae54-89a8b3cf500c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.392562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676042f0-29e3-4403-ae54-89a8b3cf500c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.392840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676042f0-29e3-4403-ae54-89a8b3cf500c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.394341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 419/844] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:47.400554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.401033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.401404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.418840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:47.420196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:47.427375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:47.443977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400929f-f49a-4e8a-980e-1f1ada61d313 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.480623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.481430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.481708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.481981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.489291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.490766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:47.506488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:47.507501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:47.508532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 426/846] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:47.509633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 420/846] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:47.515428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.529693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.623901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e9ea0-479d-405b-97ad-491c606917fc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.643771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746915aa-c9ed-45fd-b9c7-5b5a62188c79 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.659002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.659871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.660296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.660740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.661717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:47.662206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.664376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.672791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:47.684266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 427/847] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:47.690030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.690145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.690741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.691095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.691372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.692137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.693487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:47.694187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:47.707884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:47.709329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:47.710163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:47.714751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 421/848] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:47.715594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.809975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5122f5-633d-4b5e-b204-d12306cc45c4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.846385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.847422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.847944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.848414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.850302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3a2a-e1a5-41c6-a60b-040343b4d4bb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:47.895335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:47.896391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:47.896933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:47.897405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:47.898955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:47.899486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.901113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.906473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 422/849] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:47.912259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:47.944029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:47.945442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:47.947433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:47.991202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.059967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 428/850] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:24:48.060159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84beaba2-223c-4eb8-bf48-4f1f5307f8bf None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.062705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.103002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.104462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.104946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.105791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.106724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:48.107514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.109351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23e158-bd88-4afd-b70b-88ce2d5b5986 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.123222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 423/851] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:47 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:48.152620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c67d2-1e09-4cbf-8018-7af292c940d4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.152912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c67d2-1e09-4cbf-8018-7af292c940d4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.153192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c67d2-1e09-4cbf-8018-7af292c940d4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.158736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 424/852] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:48.160552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.160913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.161262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.180659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50189995-1473-4b31-a280-8bab9523f38a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.184591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.185990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:48.187711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:48.229301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.230635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.231148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.234579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.247101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.248937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:48.269105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:48.270100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:48.271280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:48.271280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:48.271280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:48.271280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:48.273991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 429/853] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:48.274676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 425/854] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:48.279107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.299042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.363773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.406113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465fddb-971f-4bec-ab71-599f9c0b3ff5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.439689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d978842-f862-4681-87ec-af069e7c2801 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.442036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.442731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.443012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.443286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.444087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:48.444399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.446544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.459596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 430/855] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:48.465421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.482248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.490646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.493384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.493384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.494122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.509167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:48.519522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:48.520476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:24:48.521442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:48.524314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 426/856] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:48.530211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.581557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12545778-26db-4b52-96d1-fb537c2360c3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.630860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.633070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.633659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.634096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.644328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.646912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.648810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301454a-dc37-4658-8d59-b63b68370f13 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.656562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:48.666464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:48.673940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 431/857] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:48.675136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.702510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.704140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.704865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.705316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.706485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:48.707419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.709488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0186aa-0148-4871-b9ab-47fa1500c22b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.720552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 427/858] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:48.722068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea13f-b3c7-49a4-8891-df9f20d60c3e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.722500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea13f-b3c7-49a4-8891-df9f20d60c3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.722943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea13f-b3c7-49a4-8891-df9f20d60c3e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.724583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 428/859] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:48.727541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea13f-b3c7-49a4-8891-df9f20d60c3e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.764435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.790917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecde89-8318-438e-a57a-979a5973776f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.828000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.828666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.828948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.829447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.829994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:48.830398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.831763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41927c-20c9-471b-937e-12dd4b27914c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.837145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea13f-b3c7-49a4-8891-df9f20d60c3e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:48.844748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 432/860] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:48.848334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.849103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.849622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.867109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:48.868328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:48.872641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:48.873462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:48.873749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:48.874025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:48.886061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:48.887665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:48.906333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:48.907435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:48.908371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:48.908371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:48.908371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:48.908371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:48.910818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 429/861] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:48.912274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:48.934786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:48.940301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 433/862] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:48.941116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.012438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:49.024500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601a11-a015-4eac-b754-3503ec141d59 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.051605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4d3a2-e3ff-4497-966a-93ec0408b92c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.062485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.063174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.063452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.063726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.064883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=8f0210144ae443e8a6d1b9e0fe196984) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:49.065197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.066657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.073240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 430/863] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/role_assignments?user.id=8f0210144ae443e8a6d1b9e0fe196984&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:49.073994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.100766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.102436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.103283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.103752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.104961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:49.105504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.107597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.132740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 434/864] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:49.133012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.205958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:49.228429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bfe72-b2ac-4f70-b3b0-39b7437ccd4c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.275836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.276425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.276930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.277036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.277717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.279374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.279761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a99f-2315-4e79-a48e-bed2c558d2b5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.309994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:49.321623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.322491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.322813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.323153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.331188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.332794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:49.349189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:49.350441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 14:24:49.351858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:49.355171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 435/865] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:24:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:49.357201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.365454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 431/866] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:24:49 2022] POST /identity/v3/services => generated 269 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:49.480567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f1f525-cb21-4677-8825-a11ace3e0b51 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.527388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.527991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.528277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.528557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.529412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:49.529728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.531203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.551000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 436/867] 10.222.0.23 () {56 vars in 1135 bytes} [Fri Jun 24 14:24:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:49.557305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.665239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715a988-5274-4458-9782-71ffc026dc9e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:49.701162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:49.701880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:49.702169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:49.702449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:49.731300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:49.775886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:49.777670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:49.880689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 432/868] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:49 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:24:49.886686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:49.930862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:49.997271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa16e8-1813-4116-a6c2-ce27256c4282 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.033440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.034174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.034436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.034741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.046506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.048051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:50.066717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:50.067669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:50.068554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:50.068554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:50.068554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:50.068554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:50.068554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 437/869] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:50.073921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:50.120667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:50.179698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20013589-41d7-4f82-a719-27127a0d70c1 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.215190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.215861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.216144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.216421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.217137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:50.217443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.218840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558f085-2524-4eb8-b0aa-e9940e31bdaa None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.231190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 433/870] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:50.236453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:50.303545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d94f4-b9e9-40f7-bc3c-65e5d5ff9e7c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.303864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d94f4-b9e9-40f7-bc3c-65e5d5ff9e7c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.304597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d94f4-b9e9-40f7-bc3c-65e5d5ff9e7c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.306564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 434/871] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:50.311527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.311814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.312090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.328789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:50.330296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:50.346679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:50.358074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b8fd9-e0d1-4977-a784-f2563ae2f12e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.398454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.399417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.399702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.399980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.408068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.409692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.416558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 435/872] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:50 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:50.431148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:50.432036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:50.432930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:50.435472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 438/873] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:50.439967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:50.443027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:50.481663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:50.561924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc356-f15d-4629-b99e-fdc3fbf2c106 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.568361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a696506-78ec-41be-8852-f38a3b033f32 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.599860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.600614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.601052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.601387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.602296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:50.602663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.604362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.607632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.608548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.609040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.609511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.610456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.612142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc24c31-ea0c-41a7-9600-84e689b91f27 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.617698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 439/874] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:50.623434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:50.635967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:50.723774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 436/875] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:24:50 2022] POST /identity/v3/services => generated 251 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:24:50.729703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80ae74-3cc3-4986-b422-d7f71af26974 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:50.783234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:50.784339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:50.784868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:50.785356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:50.786681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=52486f194961487ba574eebe49869a96) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:50.787278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:50.789061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78fc718-7836-48ca-b951-65f663aa8a4e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:50.796628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 440/876] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:50 2022] GET /identity/v3/role_assignments?user.id=52486f194961487ba574eebe49869a96&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:51.381395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c2c-b760-488e-9f99-609b5e2a668b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.381878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c2c-b760-488e-9f99-609b5e2a668b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.382478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d2c2c-b760-488e-9f99-609b5e2a668b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.384117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 437/877] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:51.389481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b3-9ef8-49e8-9463-8f2c6f03270c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.389906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b3-9ef8-49e8-9463-8f2c6f03270c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.390079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e63b3-9ef8-49e8-9463-8f2c6f03270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.407452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-669e63b3-9ef8-49e8-9463-8f2c6f03270c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:51.408627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-669e63b3-9ef8-49e8-9463-8f2c6f03270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:51.475675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:51.482122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 441/878] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:51.495282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ee7e-7748-42b4-94db-aa1ecc11848a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.495577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ee7e-7748-42b4-94db-aa1ecc11848a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.495852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4ee7e-7748-42b4-94db-aa1ecc11848a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.497406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 438/879] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:51.502722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.503042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.503326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.508975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4ee7e-7748-42b4-94db-aa1ecc11848a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:51.521672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:51.523071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:51.604801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 442/880] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:51.626926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:51.627354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:51.636201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4ee7e-7748-42b4-94db-aa1ecc11848a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:51.678135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:51.680034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.680317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.680591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.693214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:51.704493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:51.705440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:24:51.706295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:51.706986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 439/881] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:24:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:51.712739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:51.749674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe6a2c-a657-40b2-9c4f-b7e658e55680 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:51.780797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:51.787109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:51.788187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.789261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.789799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.790755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:51.792363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:51.805902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:51.806906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:51.807782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:51.812029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 443/882] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Jun 24 14:24:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:51.817730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:51.828087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5205d7-1aab-4365-bb16-b290655bbedb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:51.870552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:51.871251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.871532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.871807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.872556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:51.872851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:51.874631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:51.886673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 440/883] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:24:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:51.892464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:51.916031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:51.958555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2a1c5-a4dc-41e2-9d94-43987c559093 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:51.988618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:51.989306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:51.989589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:51.989867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:51.990626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:51.991051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:51.992561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:51.998954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 444/884] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:52.004270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.028901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2128ef5-e69a-4db8-8c64-835364c7082c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.069177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.079172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.085139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.085139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.085139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.097064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.099173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:52.119205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:52.120313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:52.121417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:52.121417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:52.121417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:52.121417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:52.121417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 441/885] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:52.121946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5965a2-d093-4060-ab05-787511354487 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.130567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.176565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.177151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.177834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.178111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.179816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:52.180126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.182337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.189182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 445/886] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:52.196033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.203522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.241892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5b0f0-655c-4c8e-9a18-ba3530af139e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.278146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.278874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.279156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.279431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.280246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:52.280556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.281920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.298655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 442/887] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:52.300838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.326268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab53f65-1b57-4f13-b8ff-b90db5e3cc50 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.358834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.385519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.386197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.386479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.386791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.387581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] RBAC: Adding query filter params (interface=public, service_id=f769ee81069f4055ae54e281cbce77bf, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:52.387894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.389295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.397008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 446/888] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/endpoints?service_id=f769ee81069f4055ae54e281cbce77bf&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:52.402770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.421587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ff7b1-2573-4ae8-b7bb-0fe37d981033 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.459077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.463571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.464329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.464680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.464957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.473333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.475265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:52.492987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:52.494007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:52.494992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:52.498774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 443/889] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:52.500722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.516174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f30f39-612c-4055-a96e-bc47bbbe95e8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.553124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.554785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.555249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.555701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.556568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.558115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.569359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 447/890] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:24:52.615179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f83d3-0304-4752-84d1-7c4a8cf4f004 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.655734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.656425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.656717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.657050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.657901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:52.658260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.659759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd69fa7-6b9c-461f-bb13-544f1608a038 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.688017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 444/891] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:52.691744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:52.748332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.801149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe6c81-520d-46a0-9da9-d6773b4dc121 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:52.836669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:52.837329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.837612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.837887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.908494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:52.910316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:52.917357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b23a-9758-4888-8fba-d99d2adf60e2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.917651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b23a-9758-4888-8fba-d99d2adf60e2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.917933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b23a-9758-4888-8fba-d99d2adf60e2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.919703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 445/892] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:52.924934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8437-cc32-4a45-8c53-6d09568886a2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:52.925194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8437-cc32-4a45-8c53-6d09568886a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:52.925476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d8437-cc32-4a45-8c53-6d09568886a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:52.943198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-791d8437-cc32-4a45-8c53-6d09568886a2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:52.944536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-791d8437-cc32-4a45-8c53-6d09568886a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:52.964549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:52.994871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.019978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 446/893] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:53.021071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa8bc-6f7a-4477-a645-d69b204f3bc4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.021349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa8bc-6f7a-4477-a645-d69b204f3bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.021618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaa8bc-6f7a-4477-a645-d69b204f3bc4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.023218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 447/894] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:53.028497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.028772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.029126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.029340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 448/895] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:24:52 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:24:53.034024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.049358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:53.050905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:53.125957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 448/896] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:53.126836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.149503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7efb9-81cd-4e3d-aef7-f6bbaa56313d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.186905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.187992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.188272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.188539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.199698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.201149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:53.218598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:53.219654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:53.220735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:53.220735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:53.220735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:53.220735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:53.224543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 449/897] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:53.225375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.228054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.236793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba7fd5-6140-4630-836e-5e91bff0d4c8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.244960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.274367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.275148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.275430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.275708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.276350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.277822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:53.289691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:53.290743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:53.291661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 449/898] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:53.292486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.336386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85186d8f-6652-4052-be1e-f599b6c3a79b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.379151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.379956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.380234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.380561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.381213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.383370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:53.395502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:53.396497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:53.398457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 450/899] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:53.399330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.408502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb2d2b-2b7c-49e5-8746-23c6a1b9c967 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.446235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.447202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.447492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.447929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.448643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.448953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.450326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.463636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 450/900] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.468942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.469710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.499646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.508764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8aced-00c7-4d46-bdbe-9a78cd0db405 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.543614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.544560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.544859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.545134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.545833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.546137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.547536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.552736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 451/901] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.558212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.580605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefce5c0-4df7-4e54-af19-1748ebb47b12 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.622225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.623267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.623553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.623831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.624526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.624837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.626217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.635709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 451/902] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.635709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.666990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98073a97-fe21-489e-bbf5-b6e1ed2c0488 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.698791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.707820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.708621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.708988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.709262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.717591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.719088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:53.733977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:53.735179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:53.736161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 452/903] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:53.736650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.744409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:53.752211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e04650-201e-4f24-83cb-ac1b315a4212 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.790717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.791422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.791705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.791981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.792687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.792995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.794430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.805078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 452/904] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.805078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.858973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a06a76-3924-438d-9399-2276b0cbd442 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.896589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.897420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.897705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.897985 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.898723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.899039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.900404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.904927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 453/905] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.905700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.918817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539e912e-f80b-406f-8f9c-c7fb9adc3c8c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:53.959018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:53.960084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:53.960835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:53.961293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:53.962368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:53.962898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:53.964584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:53.978137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 453/906] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:53.985404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:53.993964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:54.009869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:54.043476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f7de1-bb95-4ebe-a3a1-1724b416322c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:54.082778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:54.083336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.083613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.083928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.084717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] RBAC: Adding query filter params (interface=public, service_id=9dbca63e644c4d39a680eceef445a9d7, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:54.085550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:54.087093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:54.093478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 454/907] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/endpoints?service_id=9dbca63e644c4d39a680eceef445a9d7&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:54.099433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:54.106025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badec5c-bfee-4c73-99a8-1e39f85d8f4d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:54.149823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:54.150574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.151249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.151540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.152661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] RBAC: Adding query filter params (role.id=eed87914f5854e68a9fd43ab34eb3e9d, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=8f0210144ae443e8a6d1b9e0fe196984) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:54.152984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:54.154774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd170182-11c7-49da-b1aa-f72ccd892233 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:54.162766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 454/908] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:24:53 2022] GET /identity/v3/role_assignments?user.id=8f0210144ae443e8a6d1b9e0fe196984&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=eed87914f5854e68a9fd43ab34eb3e9d => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:54.220142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00edb7-33ee-42c4-974b-a7ebcc487e9c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:54.240953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:54.260686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:54.261648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.262083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.262505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.263363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:54.265012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3395cff7-e157-4ddc-a6ed-4d0d683654b3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:54.272114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 455/909] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:24:54 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:24:54.706114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86758bd-1e8f-41d9-9374-9e33ce59adba None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.706625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86758bd-1e8f-41d9-9374-9e33ce59adba None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.706625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86758bd-1e8f-41d9-9374-9e33ce59adba None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.708369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 455/910] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:54.713981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.714390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.714584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.731890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:54.733065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:54.807560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 456/911] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:54.829775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86758bd-1e8f-41d9-9374-9e33ce59adba None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:54.918408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:54.937977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86758bd-1e8f-41d9-9374-9e33ce59adba None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:54.973908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:54.974672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:54.974954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:54.975228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:54.975943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:54.977417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:54.989803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:54.990802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:54.991817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 456/912] 10.222.0.23 () {56 vars in 1132 bytes} [Fri Jun 24 14:24:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:54.997400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:55.096676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f5e9-e332-40d4-929a-798d607917f8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:55.110642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:55.131818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:55.132598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:55.132882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:55.133157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:55.133871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:55.134181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:55.135661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:55.140995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 457/913] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:55.151682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:55.257587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225ee34-d1c9-4804-ac07-c2dd1025ece2 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:55.293728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:55.294429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:55.294818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:55.295095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:55.295822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:55.296130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:55.297542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c069b35-cdbb-4a56-8ec4-8d985882e8fb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:55.303858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 457/914] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:55.309375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:55.417522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f040c62-2085-447f-b954-c81d74fde4b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:55.456421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:55.457080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:55.457357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:55.457686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:55.458485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:55.460031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0417a261-59ee-4c34-a5aa-9a715a8705ae None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:55.543458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 458/915] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:24:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:56.112183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ea11-b154-4486-be37-6855943b9660 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.112699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ea11-b154-4486-be37-6855943b9660 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.112992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6ea11-b154-4486-be37-6855943b9660 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.114485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 458/916] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:56.120581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ef9a-18fa-4e49-af73-89d70e440e83 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.121002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ef9a-18fa-4e49-af73-89d70e440e83 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.121202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47ef9a-18fa-4e49-af73-89d70e440e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.139059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1c47ef9a-18fa-4e49-af73-89d70e440e83 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:56.140244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1c47ef9a-18fa-4e49-af73-89d70e440e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:56.169253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:56.219485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 459/917] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:56.244172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6ea11-b154-4486-be37-6855943b9660 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.250940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915ef7-4c28-44ff-80ef-2e1eff71675e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.251233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915ef7-4c28-44ff-80ef-2e1eff71675e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.251522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915ef7-4c28-44ff-80ef-2e1eff71675e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.253196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 460/918] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:56.258203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:56.260403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.260690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.260969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.278567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:56.279777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:56.368465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 461/919] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:56.377439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6ea11-b154-4486-be37-6855943b9660 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:56.392850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.414287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:56.415002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.415281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.415557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.416272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:56.417637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:56.488256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:56.499635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1c3d7-1e86-4d46-92c2-ab1517a25de0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:56.532292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:56.533095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.533372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.533641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.543899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-dab40f07-154f-4417-a656-37c6ad418660 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:24:56.552430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:56.553647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 462/920] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:24:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:56.559790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3cbba-9f04-4e78-b8d7-493382cdeebf None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.561073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3cbba-9f04-4e78-b8d7-493382cdeebf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.561073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3cbba-9f04-4e78-b8d7-493382cdeebf None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.561073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 463/921] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:56.562136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64519-14e0-4603-b150-073da455614c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.562425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64519-14e0-4603-b150-073da455614c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.562738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a64519-14e0-4603-b150-073da455614c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.579471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-57a64519-14e0-4603-b150-073da455614c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:56.580676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-57a64519-14e0-4603-b150-073da455614c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:56.611850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 459/922] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:24:56 2022] POST /identity/v3/services => generated 241 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:56.612414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.669588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 464/923] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:56.692896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a64519-14e0-4603-b150-073da455614c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.753056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:56.767936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f58ba2-c068-4379-a4f7-756ec883db9e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:56.786608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:56.814312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:56.815174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.815457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.815869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.816470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:56.816775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:56.818141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:56.820482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a64519-14e0-4603-b150-073da455614c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:56.828946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 460/924] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:24:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:56.837433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.869998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:56.870850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:56.871185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:56.871466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:56.872128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:56.873734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:56.886448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:56.887573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:56.888416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 465/925] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:56.896164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:56.958855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11b1de-e8b8-4e86-8db2-5770c8ec2c06 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:56.993741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.000368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.001735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.002331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.003027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.017275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.020096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.027153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bb18e-9630-44e2-b586-5d622a30472d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:57.042564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:57.043666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:57.044640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:57.044640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:57.044640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:57.044640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:57.046908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 461/926] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:57.052271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.076024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.076690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.076972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.077245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.078050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:57.078359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.079859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.089610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 466/927] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:57.090912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.093711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.171333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e3b6a-9ad3-4db5-a4d1-996d6f87241d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.200678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf720c6-fb6d-47c6-b5cc-53e2595eab15 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.210485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.211478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.211977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.212544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.213471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:57.213981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.215589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.228936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 462/928] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:57.235105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.240051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.240323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.240546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.241586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.242199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:57.242508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.243915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.253748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 467/929] 10.222.0.23 () {56 vars in 1114 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:57.255119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.295399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.321506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.348652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bba9d7-22cf-45b6-9ae9-85c9a0585d30 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.369236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb3861-6c92-4e13-a7e0-15c6fa058115 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.392029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.393315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.394015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.394445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.405497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.406055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.406850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.407224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.407593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.408008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.408519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.409977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:24:57.426848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:57.428059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:24:57.429794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:57.431812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 463/930] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:24:57.437828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.530748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 468/931] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:24:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:24:57.556193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22fee3-9255-47b6-aef8-6fbc6bf1a3c8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.559515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.605492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.606627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.607186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.607656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.608823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:57.609576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.611349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd402fa-5b54-4a18-b636-38b4fdfccfa6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.633407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 464/932] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:57.639077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.655714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0051-7497-4519-ba8d-a548f6c97886 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.656311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0051-7497-4519-ba8d-a548f6c97886 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.656748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea0051-7497-4519-ba8d-a548f6c97886 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.659569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 465/933] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:24:57.666450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.666802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.667033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.687357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:57.688161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:57.767237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fa645-0732-4aa9-a207-a6bd7d592917 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.773995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 466/934] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:57.799132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.801526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.803805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.804779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.805061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.805448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.818316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.820679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:24:57.845116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:57.852305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 469/935] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:57.853242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:57.920686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:57.934590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b0715c-41ff-4f2e-a36c-c30a32771565 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.972776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:57.973935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:57.974416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:57.974930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:57.975880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:57.977528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:57.984030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458cc1d-2aa1-4aa0-972f-a2112a8ce55f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:57.989873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:57.990986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:24:57.991822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:24:57.999256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 467/936] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:58.000355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:58.024049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.024717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.025434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.025709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.026657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:58.027203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.028744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.044524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 470/937] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:58.050356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:58.091383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:58.112551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e36c7-15f0-4a6d-8af9-4ee9a218a067 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:58.153769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.154849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.155360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.155960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.156952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:58.157461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.159083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.165551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 468/938] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:58.179996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:58.184271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3def9-6aff-4cab-b598-f5714703e66d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:58.190037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:58.232370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.233525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.233934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.234352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.244442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.246649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:24:58.265852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:58.266907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:24:58.267802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:58.268904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 471/939] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:58.275490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:58.287843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a3078-a522-4cd0-a5b6-483b1ab423cd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:58.324655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.325277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.325562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.325840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.326573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:58.326884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.328388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.334608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 469/940] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:24:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:24:58.402493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:58.410166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e08a1-ac92-4d7e-bcd5-ef8869b8bb98 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:58.448974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.450085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.450598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.451071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.452067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:58.452574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.454259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf9cdc4-87c6-4a3c-9ee3-6a200c83e6ad None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.473741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 472/941] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:24:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:58.476142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:58.570963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:58.591304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e248340-f889-4e7f-bec8-4d0010ca25ed None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:58.632084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:58.632976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.633259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.633531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/8f0210144ae443e8a6d1b9e0fe196984/roles/eed87914f5854e68a9fd43ab34eb3e9d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.635907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=8f0210144ae443e8a6d1b9e0fe196984, role_id=eed87914f5854e68a9fd43ab34eb3e9d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:58.637352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa170e4a-ff49-4395-a385-62fc7868613c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:58.705972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 470/942] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:24:58 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/8f0210144ae443e8a6d1b9e0fe196984/roles/eed87914f5854e68a9fd43ab34eb3e9d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:24:58.868421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e06db-d5a5-46b7-ae7c-23f69ed5ae53 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.868924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e06db-d5a5-46b7-ae7c-23f69ed5ae53 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.869364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e06db-d5a5-46b7-ae7c-23f69ed5ae53 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.871259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 473/943] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:58.876343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:58.876785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:58.877160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:58.895381 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:58.896585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:58.967962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 471/944] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:24:58.991661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e06db-d5a5-46b7-ae7c-23f69ed5ae53 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.102439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e06db-d5a5-46b7-ae7c-23f69ed5ae53 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:59.139345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:59.140472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.140965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.141440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.142287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:59.143863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:24:59.156141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:24:59.157137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:24:59.157970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:24:59.160811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 474/945] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:24:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:24:59.167924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.239629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:59.271262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3fb06-0cee-4016-8075-a1ace4efaf4a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:59.307031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:59.307700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.307984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.308258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.308992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:59.309297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:59.310696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:59.315927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 472/946] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:24:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:59.322313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.342331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:59.439149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769de9e-1784-447f-9103-2522d1664297 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:59.473951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:59.474655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.475489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.475772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.476485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:59.476786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:59.478156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:59.483597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 475/947] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:24:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:59.489459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.595582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730f1a8-33a0-4fa6-ae69-5f0e15f4a04e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:59.631129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:59.631895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.632178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.632455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.633245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] RBAC: Adding query filter params (interface=public, service_id=967deff0c34a401c924b985d13cd8483, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:24:59.633560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:59.634979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85873ec7-78ee-45b6-b2e9-fcd3873cb3f7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:59.636480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:59.640600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 473/948] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:24:59 2022] GET /identity/v3/endpoints?service_id=967deff0c34a401c924b985d13cd8483&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:24:59.646193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.699082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf824-c0e6-4215-87c7-1e5c9f65b2e0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.699264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf824-c0e6-4215-87c7-1e5c9f65b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.699545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf824-c0e6-4215-87c7-1e5c9f65b2e0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.701148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 474/949] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:24:59.705808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.706096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.706374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.726110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:24:59.727755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:24:59.742001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:24:59.758965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb145c-e1c2-47bb-b1b1-08c6efb894b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:24:59.796786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:24:59.797847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:24:59.798699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:24:59.799150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:24:59.800094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:24:59.801625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:24:59.818576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 475/950] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:24:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:24:59.839844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 476/951] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:24:59 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:24:59.844950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:24:59.976474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bbde0-d4e5-46b8-b9d2-48cf595968fa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:00.005243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:00.013319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:00.013986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.014269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.014574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.015272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:00.016721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:00.018094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING keystone.server.flask.common [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 14:25:00.073532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.sql.core [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 14:25:00.073532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 14:25:00.073532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [parameters: {'id': '9ae1f31395324d74b63f896ef74e1c28', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 14:25:00.073532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80353) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ae1f31395324d74b63f896ef74e1c28.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ae1f31395324d74b63f896ef74e1c28. Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:25:00.081705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 14:25:00.085169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application yield resource Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise self.value Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application session.commit() Jun 24 14:25:00.086135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.session.flush() Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application compat.raise_( Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 14:25:00.087057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 14:25:00.087987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application result.read() Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:25:00.089641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [parameters: {'id': '9ae1f31395324d74b63f896ef74e1c28', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:00.091714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:00.093551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9ae1f31395324d74b63f896ef74e1c28. Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:00.097523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 476/952] 10.222.0.23 () {60 vars in 1140 bytes} [Fri Jun 24 14:24:59 2022] POST /identity/v3/projects => generated 262 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:00.098971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:00.211549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd682aec-55a9-4bcc-8496-3917bf9122bd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:00.281446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:00.282314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.283528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.283528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.292842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:00.294536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a90bee2b-b634-44d7-88bd-c8924ad2898d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:25:00.312776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:00.313801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:00.314803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:00.323998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 477/953] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:00.328695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:00.445104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaee621-08e7-4ee1-8f25-3e30c240bd73 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:00.483322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:00.484279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.484841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.485295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.486222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:00.486726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:00.488557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25653cc-f525-4137-86dd-e38aa3ecebec None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:00.511751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 477/954] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:25:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:00.539935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36dc8-e763-47b8-9efe-c5c0b83abd62 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.541327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36dc8-e763-47b8-9efe-c5c0b83abd62 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.542561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba36dc8-e763-47b8-9efe-c5c0b83abd62 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.546398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 478/955] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:00.552711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e285-0c23-4798-92a8-91182b53610a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.553536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e285-0c23-4798-92a8-91182b53610a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.554032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990e285-0c23-4798-92a8-91182b53610a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.580415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b990e285-0c23-4798-92a8-91182b53610a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:00.583254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b990e285-0c23-4798-92a8-91182b53610a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:00.630793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:00.662548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 478/956] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:00.698172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba36dc8-e763-47b8-9efe-c5c0b83abd62 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:00.710755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:00.838256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba36dc8-e763-47b8-9efe-c5c0b83abd62 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:00.873893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:00.874774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.875054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.875326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.876057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:00.877426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:00.946202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d1d2e-f41a-4229-a1bc-15bd283be96f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.946556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d1d2e-f41a-4229-a1bc-15bd283be96f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.946811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d1d2e-f41a-4229-a1bc-15bd283be96f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.948357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 479/957] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:00.953396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:00.953683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:00.953955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:00.963158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 479/958] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:25:00 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:00.982688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:00.984269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:01.078335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 480/959] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:01.106178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:01.187685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:01.226777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10bcf3-d49d-437a-8194-49da289cdb3d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:01.263671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:01.264537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:01.264824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:01.265086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:01.275761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:01.284594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:01.285548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:01.286379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:01.287231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 480/960] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:01.295823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:01.396940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:01.400886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902ed48-064e-48b9-af58-46f1e3fc0c6b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:01.436152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:01.436820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:01.437103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:01.437377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:01.438589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:01.439071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:01.440534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:01.450343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 481/961] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:01.455901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:01.560002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825b584-1b9b-455f-af3a-ea1ff4fd7f9b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:01.588281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:01.597606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:01.601677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:01.601677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:01.601677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:01.610866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:01.612326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:01.633828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:01.634788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:01.635730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:01.635730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:01.635730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:01.635730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:01.635730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 481/962] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:01.640993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:01.753394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240b3ec-23f3-42f8-a42e-c83536eadead None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:01.790321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:01.791422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:01.791891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:01.792361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:01.793273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:01.793763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:01.795352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:01.809507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 482/963] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:01.814952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:01.926278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9067cf-3eb5-4127-ac55-b2e5e028d80c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:01.960956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:01.962076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:01.962567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:01.963101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:01.971377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:01.973067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f6343404-7832-4323-947a-a5002abbd7bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:25:01.988239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:01.989198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 14:25:01.990154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:01.993000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 482/964] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:01.999899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.031391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.110909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397ff4-b277-4aa3-9429-433cfcb610ea None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.111233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23893b30-2600-4697-a11b-bc771b51c252 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.111338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397ff4-b277-4aa3-9429-433cfcb610ea None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.111449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397ff4-b277-4aa3-9429-433cfcb610ea None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.113249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 483/965] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:02.122857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.123137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.123414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.142497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:02.143700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:02.154392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.155552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.155844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.156127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.157537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:02.157998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.159741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.177507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.185748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 483/966] 10.222.0.23 () {56 vars in 1149 bytes} [Fri Jun 24 14:25:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:02.191531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.225361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 484/967] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:02 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:02.249288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.298035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.318504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27588c00-6a23-49ac-ac09-1d96f6687e7c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.367135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.367802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.368324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.368794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.390424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20588fa7-db5d-4e6b-a773-9807744b41d1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.426606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.427956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.428518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.428999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.429871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.431464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:02.444683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:02.445997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:02.446884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:02.454498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 485/968] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:02.455364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.456291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.456832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.489602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.558047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.578129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36f802-d5ce-4e07-af05-780eb845d8f1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.582921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 484/969] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:02.595513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.618855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.619682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.620057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.620425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.621391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:02.621763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.623224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.629168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 486/970] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:02.635146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.700101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ecef1-0543-4da2-9b06-1ec09b740d99 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.742556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.743513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.744320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.744603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.744876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.752726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4543a2-dc67-4115-b720-39215df982f3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.757841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.761360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:02.778794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:02.779723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:02.780621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:02.780621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:02.780621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:02.780621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:02.784956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 485/971] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:02.791586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.800987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.802047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.802506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.803373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.804549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:02.805037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.807072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.813968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 487/972] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:02.819096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.826852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:02.921449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633b7fc-86ab-44c8-b1dc-71e4fa5503b4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.940353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a6708e-b953-40e6-b21e-9a22e37ce74f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:02.958029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.958622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.958904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.959735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.960452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:02.960764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.962113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6044371-9736-4a9e-a5c4-053e150ea6cf None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:02.978546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 486/973] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:02.978546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd96f0-4dd1-4697-8747-8688f634a44b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.978546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd96f0-4dd1-4697-8747-8688f634a44b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.978546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd96f0-4dd1-4697-8747-8688f634a44b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.978546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 487/974] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:02.981025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd96f0-4dd1-4697-8747-8688f634a44b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:02.982449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:02.983179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:02.983469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:02.983749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:02.985531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:02.986973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464c659-952c-4d96-9ea4-b6dd425f04fa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.074505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.084997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd96f0-4dd1-4697-8747-8688f634a44b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.087965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 488/975] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:25:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:03.089265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6655-6406-4394-8908-d5bd19a91571 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.089558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6655-6406-4394-8908-d5bd19a91571 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.089829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce6655-6406-4394-8908-d5bd19a91571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.112229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5fce6655-6406-4394-8908-d5bd19a91571 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:03.113620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5fce6655-6406-4394-8908-d5bd19a91571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:03.127626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.137539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.138791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.139335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.139799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.147793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.149611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:25:03.166915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:03.168274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:03.169200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:03.171657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 488/976] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:03.177679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.207016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 489/977] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:03.214381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45defe60-8eb2-4a18-932e-058d135244c0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.214704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45defe60-8eb2-4a18-932e-058d135244c0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.215095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45defe60-8eb2-4a18-932e-058d135244c0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.217026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 490/978] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:03.222404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.222850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.223289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.242077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:03.243317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:03.304600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f29d0-cda4-41cf-96d6-d9b92913e4a6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.328703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 491/979] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:03.329839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.348199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.349118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.349409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.349688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.350504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:03.351334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.352945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.375987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 489/980] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:03.375987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.388220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.419237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.441956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba3e4a1-97a0-43f2-bc46-cdb871459695 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.476822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.477434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.477700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.477939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.478660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.480102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.481403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING keystone.server.flask.common [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 14:25:03.481798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21792b9a-7d3b-4bca-883f-08fbff145f8e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.516742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.517530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.517815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.518091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.518766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.520136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.529434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.sql.core [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 14:25:03.529434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 14:25:03.529434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [parameters: {'user_id': '200714871ba84ca5accd8a3443858d3b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 14:25:03.529434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80352) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:03.531653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:03.532651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:03.533535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 490/981] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:03.535049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result.read() Jun 24 14:25:03.553787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:25:03.554970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application yield resource Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application next(self.gen) Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise self.value Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application session.commit() Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 14:25:03.555966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.session.flush() Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application compat.raise_( Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 14:25:03.556953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 14:25:03.557852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application conn.query(q) Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result.read() Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 14:25:03.558761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [parameters: {'user_id': '200714871ba84ca5accd8a3443858d3b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:03.559838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 14:25:03.560725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 492/982] 10.222.0.23 () {60 vars in 1131 bytes} [Fri Jun 24 14:25:03 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:03.562179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.618750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.651482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e80b-4abc-4bc5-9581-c071be4754d4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.669263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766cf808-802d-431a-b082-a1502a94d7a2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.687486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.688149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.688432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.688711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.689533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] RBAC: Adding query filter params (role.id=eed87914f5854e68a9fd43ab34eb3e9d, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=8f0210144ae443e8a6d1b9e0fe196984) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:03.689845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.691301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.706546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 491/983] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/role_assignments?user.id=8f0210144ae443e8a6d1b9e0fe196984&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=eed87914f5854e68a9fd43ab34eb3e9d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:03.706546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.720362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.721209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.721585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.721953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.722880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:03.723816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.726977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.743062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 493/984] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:03.745186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.770897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.820054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62c2f5-1726-4595-bb41-f8fae9259456 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.869046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.869737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.870016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.870283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.873301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:03.879532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.880447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579cccf8-56ca-4502-a520-3e6336ef51e2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:03.886123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:25:03.897979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:03.899247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 14:25:03.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:03.904173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 492/985] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:03.905047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:03.926726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:03.927915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:03.928444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:03.929311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:03.930281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:03.930823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:03.932867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:03.940045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 494/986] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:03.946428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:04.018426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694c86ef-d8c7-4fe4-9aff-932d4d2dea1d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:04.019195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:04.055500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:04.056359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:04.056750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:04.057102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:04.057328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e05625-64a1-435f-b18c-72b6994efd48 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:04.058049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:04.058425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:04.059969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:04.093601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:04.094643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:04.095105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:04.095559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:04.096515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] RBAC: Adding query filter params (interface=public, service_id=631c1c5ffce64e638fe4f717851cde09, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:04.097130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:04.097532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 493/987] 10.222.0.23 () {56 vars in 1101 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:04.099143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64fc7ab-52c6-4516-8a09-dad29ad93772 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:04.105514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 495/988] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:25:03 2022] GET /identity/v3/endpoints?service_id=631c1c5ffce64e638fe4f717851cde09&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:04.114474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:04.194938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:04.231044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfa04d-fa7b-4319-bd80-0eb2815a57c7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:04.274486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:04.275558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:04.276100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:04.276480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:04.277277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:04.278791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0772cfec-74d6-4fbf-a1d5-f66913c0751e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:04.285953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 494/989] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:25:04 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:25:05.185272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611c7-530b-4feb-8b26-44846bc323a7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.185800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611c7-530b-4feb-8b26-44846bc323a7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.185800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e611c7-530b-4feb-8b26-44846bc323a7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.187254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 496/990] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:05.192220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f08b1-acb4-4576-ac97-57693633978c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.192605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f08b1-acb4-4576-ac97-57693633978c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.192772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f08b1-acb4-4576-ac97-57693633978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.209749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-140f08b1-acb4-4576-ac97-57693633978c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:05.210945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-140f08b1-acb4-4576-ac97-57693633978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:05.277527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 495/991] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:05.300518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e611c7-530b-4feb-8b26-44846bc323a7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:05.408046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e611c7-530b-4feb-8b26-44846bc323a7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:05.439855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:05.446910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:05.447716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.448001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.448271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.448917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:05.450284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:05.463366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:05.464601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:05.465446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 497/992] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:25:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:05.470726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f08b1-acb4-4576-ac97-57693633978c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:05.526849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:05.576830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f08b1-acb4-4576-ac97-57693633978c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:05.610321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:05.611010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.611292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.611564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.612259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:05.612564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:05.613917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:05.618945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 496/993] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:05.624264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:05.723309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bee7b1-170e-4dcf-9408-5632a22a236b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:05.757690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:05.758355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.758665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.758936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.759630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:05.759932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:05.761288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a7cfa-9067-4afd-9d5a-f2310d53cf2f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:05.766285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 498/994] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:05.774581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:05.873796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fe342-5538-4e57-89f3-873f131e2f86 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:05.885637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db9a6-7d24-4296-b2ea-ab74bdc419a3 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.885922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db9a6-7d24-4296-b2ea-ab74bdc419a3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.886195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1db9a6-7d24-4296-b2ea-ab74bdc419a3 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.887591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 499/995] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:05.890509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:05.892587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.892879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.893155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.910577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:05.912995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:05.913949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:05.914646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:05.915054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:05.915335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:05.916140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] RBAC: Adding query filter params (interface=admin, service_id=967deff0c34a401c924b985d13cd8483, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:05.916457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:05.917831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:05.924729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 497/996] 10.222.0.23 () {56 vars in 1249 bytes} [Fri Jun 24 14:25:05 2022] GET /identity/v3/endpoints?service_id=967deff0c34a401c924b985d13cd8483&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:05.930851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:05.973866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:05.989140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 500/997] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:06.011211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.036014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba3ed-0ca9-4787-bea6-2e970da0a24f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.074836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.075748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.076200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.076700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.077651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.079152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.098546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 498/998] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:25:05 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:25:06.127068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1edafe-53ea-4cc6-bbbd-b671995018ef None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.165501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.166372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.166686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.167003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.167757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.169154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:06.181936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:06.183104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:06.183957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:06.188357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 501/999] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:06.190111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.238604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:06.300448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fe910-8a17-4179-8cce-fa463f122234 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.337746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.338415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.338731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.339006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.339707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:06.340011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.341372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b2dda-50bd-4934-b6b4-5c1b5914d88d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.346664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 499/1000] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:06.352497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.373450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dadab-5cea-4fbe-880f-5ac2fd619280 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.373738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dadab-5cea-4fbe-880f-5ac2fd619280 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.374016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dadab-5cea-4fbe-880f-5ac2fd619280 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.375423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 500/1001] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:06.380578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.381554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.381830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.401811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:06.403494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:06.411797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:06.461201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa54c4-9f82-42e2-a8d5-eebdbde39ca9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.479475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 501/1002] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:06.496984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.497578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.497861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.498135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.498867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:06.499175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.500551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.502253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.505379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 502/1003] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:06.543857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:06.620060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ab2917-a506-4d2f-bb42-55958970dc7f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.657263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.658063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.658346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.658647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.659302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.660680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:06.674215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:06.675237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:06.676080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 502/1004] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:06.682850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.791136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b6eda-bfd5-4116-8a7f-715923c2c5b4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.807924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:06.825340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.826009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.826286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.826582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.827283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:06.827593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.828961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.839069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 503/1005] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:06.840652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:06.942758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:06.947611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9afb300-a24e-49ad-b6e4-5ed39aa30738 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:06.982021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:06.982761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:06.983427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:06.984046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:06.985258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:06.985569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:06.986986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20f561-b966-48e5-9dc5-7c771436ee32 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:06.992365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 503/1006] 10.222.0.23 () {56 vars in 1112 bytes} [Fri Jun 24 14:25:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:06.998063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:07.100486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc4ed4-fee3-41cf-a40f-658be9709eb9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:07.133612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:07.134268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:07.134569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:07.134847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:07.135675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:07.137059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb2173-5d90-464b-a439-dd4dc27e7575 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:07.223695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 504/1007] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:25:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:08.182836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc4710-dfe0-47ce-954c-da46eddb05c3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.183358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc4710-dfe0-47ce-954c-da46eddb05c3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.183438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc4710-dfe0-47ce-954c-da46eddb05c3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.184923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 504/1008] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:08.189947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.190240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.190606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.207664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:08.208884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:08.238620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:08.283585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 505/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:08 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:08.306965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc4710-dfe0-47ce-954c-da46eddb05c3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:08.427379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc4710-dfe0-47ce-954c-da46eddb05c3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:08.465691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:08.466406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.466719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.466994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.467648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:08.469024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c4279f72-108f-4bd6-8181-f10853f01694 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:08.482705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:08.483702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:08.484572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 505/1010] 10.222.0.23 () {56 vars in 1126 bytes} [Fri Jun 24 14:25:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:08.489838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:08.558542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a913-9d35-4796-a5f7-178c9e5e96ed None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.558850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a913-9d35-4796-a5f7-178c9e5e96ed None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.559132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220a913-9d35-4796-a5f7-178c9e5e96ed None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.560528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 506/1011] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:08.565315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.565606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.565886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.583272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:08.585250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:08.606253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:08.606418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ecbff-003d-430d-a7f4-2af38275fbc1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:08.643663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:08.644665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.644954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.645238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.645944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:08.646937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:08.648367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:08.657888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 506/1012] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:08.658755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:08.659533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 507/1013] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:08 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:08.682706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:08.689556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:08.766858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0bd4-92d0-4720-91b6-2ca132c7bd44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:08.788335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2398a-4c71-40d3-a94b-8196e70708d3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:08.800791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:08.801454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.801734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.802007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.803415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:08.803719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:08.805076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:08.809938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 507/1014] 10.222.0.23 () {56 vars in 1122 bytes} [Fri Jun 24 14:25:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:08.815485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:08.830160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:08.830939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.831224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.831663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.832227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:08.833681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912e968-777f-429e-9a9c-404e39981d67 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:08.904550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:08.914038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76a7fe-f413-464f-b1db-3df0c92d663a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:08.919234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 508/1015] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:25:08 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:25:08.958099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:08.959524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:08.959956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:08.960594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:08.961360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:08.962799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4f96a-9d93-4eb6-b446-ae604113e95b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:09.054558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 508/1016] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:25:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:09.417178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf3df-faa5-49b3-ab50-201c08a68b2d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:09.417682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf3df-faa5-49b3-ab50-201c08a68b2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:09.417787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bf3df-faa5-49b3-ab50-201c08a68b2d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:09.419278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 509/1017] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:09.424423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:09.424823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:09.425003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:09.442344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:09.444397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:09.520199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 509/1018] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:09.543608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bf3df-faa5-49b3-ab50-201c08a68b2d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:09.563353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:09.647740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bf3df-faa5-49b3-ab50-201c08a68b2d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:09.683742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:09.684580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:09.684867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:09.685145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:09.696315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:09.704502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:09.705442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:09.706296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 510/1019] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:25:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:09.713003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:09.783264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:09.813687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5af29-841d-4804-b1ca-eca20e38e788 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:09.847276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:09.847934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:09.848234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:09.848484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:09.849745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:09.850186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:09.851584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:09.864032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 510/1020] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:25:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:09.865927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:09.943054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:09.965502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1200f-d986-4e14-a381-b925bc813d3c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:09.999946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:10.000731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:10.001009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:10.001280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:10.011822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:10.013264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:10.033447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:10.034385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:10.036030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:10.036030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:10.036030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:10.036030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:10.036030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 511/1021] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:10.044596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:10.160702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544927a8-ad38-40f3-a6ef-0c26e7c5560e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:10.199529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:10.200119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:10.200402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:10.200676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:10.201399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:10.201710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:10.203122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:10.204853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:10.221347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 511/1022] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:10.222498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:10.325596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce2ec0-05a8-49e1-9239-e569b024413d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:10.350553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:10.360803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:10.365867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:10.365867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:10.365867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:10.374886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:10.376356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:25:10.391112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:10.392311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:10.393479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:10.400328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 512/1023] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:25:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:10.401412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:10.503153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0aa15-27ca-4d85-945d-2b062466c98a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:10.537902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:10.538589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:10.538870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:10.539138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:10.539964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:10.540269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:10.541686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:10.564463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 512/1024] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:25:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:10.565602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:10.666439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375068cc-653f-4180-a8fe-59339341bca0 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:10.701399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:10.702046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:10.702326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:10.702624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:10.757759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:10.770202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:10.772285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fd762-a269-48c0-9d16-a12f9e48e251 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:10.874960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 513/1025] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:10.876467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:10.931208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:10.979627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ca4b4-8ce3-437c-8622-d6db4dfafc1c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.002073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdedc0-276d-4823-bd60-15adb09c5c2a None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.002366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdedc0-276d-4823-bd60-15adb09c5c2a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.002682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdedc0-276d-4823-bd60-15adb09c5c2a None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.004064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 514/1026] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:11.010779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18190-1d5c-429c-967b-bf7b2b1673e1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.011002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18190-1d5c-429c-967b-bf7b2b1673e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.011283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee18190-1d5c-429c-967b-bf7b2b1673e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.015320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.016552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.017879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.018307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.027849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ee18190-1d5c-429c-967b-bf7b2b1673e1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:11.029030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ee18190-1d5c-429c-967b-bf7b2b1673e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:11.031197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.033648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:11.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:11.053187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:11.054090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:11.054090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:11.054090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:11.054090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:11.054090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 513/1027] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:11.059573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.107659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 515/1028] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:11.109213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f03fb-af0e-48ef-9109-921d8b612f16 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.109502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f03fb-af0e-48ef-9109-921d8b612f16 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.109778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f03fb-af0e-48ef-9109-921d8b612f16 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.111186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 516/1029] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:11.117040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.117330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.117601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.134686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:11.135902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:11.149185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.168198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbefae5-3fe5-429a-8e4e-82e47798723b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.170697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.202906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.203561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.204131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.204254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 517/1030] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:11.204336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.204739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.204835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:11.205162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.206475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.223004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 514/1031] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:11.225291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.312437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d8682b-43ca-44d3-8236-a1d5c5972a51 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.324571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db0919-89df-4316-9f63-f84a49fbedcb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.346634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.347411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.347688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.347958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.348626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.350554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.359467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.360235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.360515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.360784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:11.362155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:11.363140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:11.363982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 518/1032] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:11.365372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.371013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.372475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.381106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:25:11.388113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:11.394906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 515/1033] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:11.395991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.419440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.465534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aabff3-677a-418d-b25f-9ea4b2cde835 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.494765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552cc5-1f5d-4ebc-aa71-42cd6d572eb0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.499499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.500289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.500568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.500839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.502008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.503467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.505088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 519/1034] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:11.505088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.529307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.529975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.530256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.530539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.531286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] RBAC: Adding query filter params (name=image) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:11.531590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.532999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.541546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 516/1035] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:11.541546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.587421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.606460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ac45-126e-4232-a412-ca4bbd027a3d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.627074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.638560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5d513-40a9-4bb1-af93-1455680d4510 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.639935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.640486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.640774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.641040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.641819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:11.642127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.643570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.655276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 520/1036] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:11.656061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.672856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.673517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.673799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.674073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.674808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.676241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.754980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b0688-b96b-4a2d-96fb-4ce0bec6a2d3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.759531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 517/1037] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:25:11 2022] POST /identity/v3/projects => generated 311 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:11.760361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.802216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.803023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.803389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.803667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.804393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:11.804701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.806113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.818277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 521/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:11.819503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:11.824583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:11.918168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d593a4c-19e2-40e5-8a38-e0d2a314024a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:11.961813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:11.962481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:11.962790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:11.963059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:11.963888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] RBAC: Adding query filter params (role.id=eed87914f5854e68a9fd43ab34eb3e9d, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:11.964200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:11.966029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:11.973109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 518/1039] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=eed87914f5854e68a9fd43ab34eb3e9d => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:11.991980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3c9a4-8b08-4e62-8dac-fcd7c283b8bd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:12.030055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:12.030775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:12.031058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:12.031332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:12.032146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] RBAC: Adding query filter params (interface=public, service_id=cd66151a1c62457690818c05e4d0ce7f, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:12.032465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:12.033865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530983f3-4b6d-44cb-a08b-4a7f2cf92c21 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:12.040209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 522/1040] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:25:11 2022] GET /identity/v3/endpoints?service_id=cd66151a1c62457690818c05e4d0ce7f&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:12.046060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:12.050343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:12.158208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b98f-9a16-4499-8639-c52937eee6d7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:12.194649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:12.195533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:12.195975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:12.196411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:12.197245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:12.198834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b571c4e-43b3-4a23-bb2f-fa0952ab41bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:12.205966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 519/1041] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:25:12 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:25:13.833549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90aa7-bdc4-401f-a6c5-071705cc2a85 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:13.833549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90aa7-bdc4-401f-a6c5-071705cc2a85 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:13.834595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90aa7-bdc4-401f-a6c5-071705cc2a85 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:13.835473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 523/1042] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:13.840834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea50ea-73b2-40d6-a4a3-23b08945af23 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:13.841321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea50ea-73b2-40d6-a4a3-23b08945af23 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:13.841411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea50ea-73b2-40d6-a4a3-23b08945af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:13.858831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7bea50ea-73b2-40d6-a4a3-23b08945af23 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:13.860014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7bea50ea-73b2-40d6-a4a3-23b08945af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:13.923744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:13.930444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 520/1043] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:13.955224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c90aa7-bdc4-401f-a6c5-071705cc2a85 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:13.987810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134900cf-de7d-4757-a499-9a98cf308815 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:13.988092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134900cf-de7d-4757-a499-9a98cf308815 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:13.988373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134900cf-de7d-4757-a499-9a98cf308815 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:13.990099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 521/1044] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:13.995013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:13.995314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:13.995597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.011823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:14.013012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:14.030560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.069594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c90aa7-bdc4-401f-a6c5-071705cc2a85 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.091340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 522/1045] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:14.110600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.111400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.112206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.112618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.112773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.113967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.115591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.121681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 524/1046] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:14.124836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.218578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a8b30c-69cf-42a1-aef7-4feebb18a91a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.242385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2072e13-24b1-4923-93cb-f02c6adfbdcd None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.247736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.255894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.256867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.259213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.259213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.267557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7c96c100-2218-4fa9-a3b9-b0463c5f316f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:14.275923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:14.277077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:14.277987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 523/1047] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:14.278783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0053140-1a0d-4bd6-9eb0-cb95bda44086 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.278942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0053140-1a0d-4bd6-9eb0-cb95bda44086 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.279198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0053140-1a0d-4bd6-9eb0-cb95bda44086 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.280616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 524/1048] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:14.280724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.281360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.281717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.282183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.283056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.283496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0053140-1a0d-4bd6-9eb0-cb95bda44086 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.284702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c4965-9c2c-4d70-a813-50c5a1217796 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.293175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.387411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0053140-1a0d-4bd6-9eb0-cb95bda44086 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.420839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.421495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.421774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.422043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.423173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:14.423477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.424850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.431635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 525/1049] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:25:14 2022] POST /identity/v3/users => generated 285 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:25:14.433011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.433304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.433583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.438939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 525/1050] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:14.440151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.456196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:14.458223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:14.497293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.543776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 526/1051] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:14.556614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327133f-eeb2-4d18-9a8f-428484475ef8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.567974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.598130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.599481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.599762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.600033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.616539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.618461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:14.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:14.637644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:14.638566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:14.638566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:14.638566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:14.638566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:14.638566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 526/1052] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:14.643802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.681655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d1705f-1509-4edb-897b-b47d1e8a7056 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.705930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.721518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.722615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.722903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.723176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] PATH_INFO: `/v3/services/image` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.724744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.727075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:14.739342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:14.740477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:14.746615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 527/1053] 10.222.0.23 () {56 vars in 1105 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:14.748212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.761673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489edc1a-0546-407d-82d8-53f0bea15524 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.763358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:14.798437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.799661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.799950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.800538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.801267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:14.801583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.803002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.814728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 527/1054] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:14.820355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.874360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a269e9f-f563-4c7e-8449-e6c8654c0994 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.913269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.913941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.914226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.914506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.915244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:14.915555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.916922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.926569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 528/1055] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:14.928658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:14.936348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2bda4f-cad6-4cf8-99a0-2dea9747ff29 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:14.972716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:14.974823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:14.974984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:14.975190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:14.983668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:14.985291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:14.985438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:25:15.000197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:15.001375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:15.002563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 528/1056] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:15.007632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.033875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:15.042136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bee75d-7bac-46d3-a958-feeeb9b66ce7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.084379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.085042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.085324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.085659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.086282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] RBAC: Adding query filter params (type=image) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:15.086614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.087970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.093247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 529/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:15.098658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.135194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a2888-8e9b-4010-93e8-a8b673abb1ab None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.170979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.171654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.171933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.172457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.173344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:15.173659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.175118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.195486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 529/1058] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:25:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:15.200961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.220366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ae01a-b1ff-46b1-9bef-5d1c070794f8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.234998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:15.257478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.258144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.258428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.258737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.259546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.260920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.289222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:15.309104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28091470-7d36-48a4-b161-f2b8ff5e1a5a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.344147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.345027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.345314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.345590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.358379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.360026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:15.377875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:15.385317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:15.386241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:15.386241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:15.386241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:15.386241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:15.386241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 530/1059] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:15.386593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 530/1060] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:25:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:25:15.388779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.504563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748c11-5d18-422e-930e-192e896dd3c3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.540309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:15.555246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.555547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.555831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.556108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.557029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:15.557134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.558501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.570967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 531/1061] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:15.576589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.687892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c4273-a7a2-4378-a7a5-ff096f055ac5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.722617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.723419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.723869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.724566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.731406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.732855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:25:15.748059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:15.748937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:15.749833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 531/1062] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:15.756176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.858872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891e316-d85f-4b5e-99da-2a1d454bba9a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:15.893205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:15.893868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:15.894148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:15.894419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:15.895235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:15.895633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:15.897088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3debd-bd1b-421b-ad03-09e47d991a75 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:15.909329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 532/1063] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:25:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:15.915661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:15.946384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:16.020082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b861242-214f-4120-bff7-59e42c8c56f4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:16.055906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:16.056788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:16.057066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:16.057335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/eed87914f5854e68a9fd43ab34eb3e9d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:16.059538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=4bfc33870df145a4ba481afd9cdab92b, role_id=eed87914f5854e68a9fd43ab34eb3e9d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:16.060965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f99e6-b47a-44e5-9355-cc08eda26345 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:16.140846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 532/1064] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:25:15 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/4bfc33870df145a4ba481afd9cdab92b/roles/eed87914f5854e68a9fd43ab34eb3e9d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:25:16.541817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0108f10-81c1-40df-ad4f-069979d8cca4 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:16.542382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0108f10-81c1-40df-ad4f-069979d8cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:16.542382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0108f10-81c1-40df-ad4f-069979d8cca4 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:16.544999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 533/1065] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:16.549861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:16.550156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:16.550437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:16.568681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:16.570199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:16.642537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 533/1066] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:16.666732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0108f10-81c1-40df-ad4f-069979d8cca4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:16.720477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:16.773879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0108f10-81c1-40df-ad4f-069979d8cca4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:16.808280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:16.809078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:16.809357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:16.809628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:16.821855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:16.834460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:16.835577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:16.836446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:16.838589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 534/1067] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:16.842392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:16.865513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:16.950124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a1c6b-83f0-4e1b-8cff-538084a48591 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:16.985190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:16.985890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:16.986226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:16.986509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:16.987281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:16.987596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:16.988970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:17.002723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 534/1068] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:17.004315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:17.109378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825c0b3-dc06-4a4f-9a9b-01e7ee1bd023 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:17.119584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:17.144761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:17.146376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.146854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.147136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] PATH_INFO: `/v3/users/79edd06616304031af478e76fd08fef1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.163305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] RBAC: Authorizing `identity:get_user(user_id=79edd06616304031af478e76fd08fef1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:17.164853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:17.171872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 535/1069] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:17 2022] GET /identity/v3/users/79edd06616304031af478e76fd08fef1 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:17.173027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:17.265553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:17.286877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cd0c8-917e-4096-ab7b-53528e700d69 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:17.323377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:17.324124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.324480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.324828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.404786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:17.406395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec0cfa-03d1-4656-92c2-90a1522b2670 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:17.489296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:17.513689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 535/1070] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 14:25:17.514999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:17.557167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626e88-426f-4095-8f90-0ab213fdf068 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.557454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626e88-426f-4095-8f90-0ab213fdf068 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.557725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93626e88-426f-4095-8f90-0ab213fdf068 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.559150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 536/1071] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:17.564537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.564899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.565181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.582251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:17.584120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:17.639870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e6fca-eb47-40b1-a743-b906b295ebc9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:17.652510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 537/1072] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:17.674460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:17.680121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:17.681232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.681695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.682603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.683999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:17.686402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:17.688277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 536/1073] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:17 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:17.695426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:17.731393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:17.783419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7cb162-f3d6-428c-ad4a-0c35e862401b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:17.799599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:17.800032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103ef34-6ac3-4fd8-addf-0d024ec1c6f9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:17.824556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:17.825246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.825534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.825810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.837191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:17.837961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:17.838253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:17.838552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:17.839493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=c8b89bfd17254438a8f8944a30c704bb, user.id=79edd06616304031af478e76fd08fef1) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:17.839811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:17.841253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:17.854804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 537/1074] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:17 2022] GET /identity/v3/role_assignments?user.id=79edd06616304031af478e76fd08fef1&scope.project.id=c8b89bfd17254438a8f8944a30c704bb&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:17.916061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:17.917560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:18.025975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 538/1075] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 14:25:18.032189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:18.132831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:18.137068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec9e58-69c8-4293-b9b7-178b8f9f48a3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:18.170629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:18.172244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.172777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.173364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.184602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:18.186057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2ed51cba-f563-432f-aa09-2653b7bc7a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:18.204642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:18.205563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:18.206462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:18.206462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:18.206462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:18.206462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:18.206462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 538/1076] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:18.213566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:18.228385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:18.323085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661697d2-f91a-4910-8fb9-f4214531d1d7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:18.324396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1c70-3b4e-4eb9-99e3-421a5b663734 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.324693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1c70-3b4e-4eb9-99e3-421a5b663734 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.324950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e1c70-3b4e-4eb9-99e3-421a5b663734 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.326452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 539/1077] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:18.334444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.334821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.335013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.352349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:18.353846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:18.367475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:18.368151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.368573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.368856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.369740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:18.369870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:18.371273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:18.394644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 539/1078] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:18.445966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 540/1079] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:18.470741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:18.547786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:18.579835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82624971-7c9e-4fb7-928a-8ebcecd9667e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:18.615687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:18.616509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.616786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.617057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.627531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:18.635866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:18.636810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:18.637680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 540/1080] 10.222.0.23 () {56 vars in 1120 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:18.642972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:18.725573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:18.743521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86e917-4173-4aac-bfff-ee9b3ef937d7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:18.776595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:18.777391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.777670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.778048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.778809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:18.779114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:18.780467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:18.788819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 541/1081] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:18.794415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:18.891001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e2788-fccd-474f-b3b7-873ff39191ad None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:18.918999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:18.924131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:18.925000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:18.925351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:18.925694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:18.936437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:18.937888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:18.955328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:18.956237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:18.957129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:18.957129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:18.957129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:18.957129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:18.958847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 541/1082] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:18.962973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.061050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cbc0a-17ab-4fcc-8dde-0a46d9db871f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:19.093219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:19.093867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.094147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.094419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.095152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:19.095457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:19.096835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:19.108890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 542/1083] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:19.114990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.216824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbd199-3d36-41b1-9e58-5c2e2ff6b942 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:19.248624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:19.249390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.249668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.249940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.257156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:19.258608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:25:19.272723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:19.273825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:19.274847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 542/1084] 10.222.0.23 () {56 vars in 1093 bytes} [Fri Jun 24 14:25:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:19.279508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.282299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:19.382662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d6ad9-7d36-4028-b00f-9551b5ea2c8b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:19.416613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:19.417277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.417550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.417818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.418668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:19.418976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:19.420402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:19.437450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 543/1085] 10.222.0.23 () {56 vars in 1133 bytes} [Fri Jun 24 14:25:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:19.443135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.488003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:19.543925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a654f3-941c-4578-9758-d5596a42ca4a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:19.576269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:19.576910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.577187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.577459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.644893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:19.647078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:19.689245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:19.742425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 543/1086] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 14:25:19.748151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.807767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:19.848552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115268d1-cdde-448b-b71a-54c4cede4b7f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:19.880644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:19.881429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.881706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.881975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.892294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:19.893726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a563cb87-764f-443a-987a-8ac46b6e7448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:19.911311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:19.912408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:19.913326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:25:19.913326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:19.913326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:25:19.913326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:19.913326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 544/1087] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:19.918076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:19.925448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4553dc-196d-492e-b450-5550ad7b61f5 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.925740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4553dc-196d-492e-b450-5550ad7b61f5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.926022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4553dc-196d-492e-b450-5550ad7b61f5 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.928118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 545/1088] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:25:19.933498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:19.933792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:19.934078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:19.951895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:19.953096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:20.020494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 546/1089] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:20.021739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fa68b-e229-43f7-870d-6ff3575146e3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.043030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.058322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.059024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.059310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.059587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.060335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:20.060644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.062019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.074335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 544/1090] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:20.079827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.084136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.085381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.140904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8299-e1cb-4d7d-8be2-89be1ccc6c3c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.177311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.178126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.178405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.178699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.181592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fdbc7-ba3b-49e9-8ebd-54ebc1c7318c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.188532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:20.196903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:20.197950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:20.198890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 547/1091] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:20.204335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.215636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.216430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.216718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.216997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.224170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.225634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:25:20.240548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:20.241606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:20.242923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 545/1092] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:20.247574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.300279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d7956-c634-4b4e-bfcd-4bae9f2d1f96 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.302405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.309825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.334477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.335199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.335477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.335746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.336481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:20.336782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.338151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.346861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 548/1093] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:20.353142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.354955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ff807-164e-4cf2-8126-76a12c310ab7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.390443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.391223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.391507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.391785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.392565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:20.392885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.394320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.405722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 546/1094] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:20.414142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.461222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa0cc1-cd3f-4223-b0ec-e2b912ee5de3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.494450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.495347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.495630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.495905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] PATH_INFO: `/v3/users/79edd06616304031af478e76fd08fef1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.497089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] RBAC: Authorizing `identity:get_user(user_id=79edd06616304031af478e76fd08fef1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.498531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.500161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 549/1095] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/users/79edd06616304031af478e76fd08fef1 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:20.505813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.518331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c398d-49fe-47de-a6ad-069e62169fa7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.536362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.542227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:20.554354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.556892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.556892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.556892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.556892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] RBAC: Adding query filter params (role.id=eed87914f5854e68a9fd43ab34eb3e9d, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=4bfc33870df145a4ba481afd9cdab92b) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:20.556892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.558284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.564167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 547/1096] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/role_assignments?user.id=4bfc33870df145a4ba481afd9cdab92b&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=eed87914f5854e68a9fd43ab34eb3e9d => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:20.609128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf442c3-9dbf-4b1d-bf98-d37b2d3ba73f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.646392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.647307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.647586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.647859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.649091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.650589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12bcb4-c9f9-4b57-9227-29437918c85b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.652335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 550/1097] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:20 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:25:20.658419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:20.768255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64198d1f-04a9-451c-816b-94cd19a347e9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:20.797936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:20.798859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:20.799145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:20.799429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb/users/79edd06616304031af478e76fd08fef1/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:20.801536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] RBAC: Authorizing `identity:create_grant(project_id=c8b89bfd17254438a8f8944a30c704bb, user_id=79edd06616304031af478e76fd08fef1, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:20.803095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881327aa-dc14-4471-b06b-5483503ca748 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:20.860150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 548/1098] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:25:20 2022] PUT /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb/users/79edd06616304031af478e76fd08fef1/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:25:22.883959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961241f3-85fd-4c40-a8a7-f860a84b4c1d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:22.884483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961241f3-85fd-4c40-a8a7-f860a84b4c1d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:22.884577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961241f3-85fd-4c40-a8a7-f860a84b4c1d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:22.885885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 551/1099] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:22.891084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:22.891508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:22.891644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:22.908233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:22.909447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:22.952975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:22.976940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 549/1100] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:23.004414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961241f3-85fd-4c40-a8a7-f860a84b4c1d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.103173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:23.106909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961241f3-85fd-4c40-a8a7-f860a84b4c1d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:23.138504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:23.139249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:23.139731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:23.140214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:23.150890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:23.158463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:23.159560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:23.161204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 552/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:23.166380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.261487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161dcf6-66ed-454d-ad29-917b816949f6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:23.294927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:23.295585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:23.295866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:23.296159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:23.305100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:23.305100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:23.305100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:23.307718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 550/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:23.313353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.411313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10887338-6a44-435c-a2da-2efa8b70958a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:23.444183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61092361-2641-4c32-a283-bec782223706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:23.445046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092361-2641-4c32-a283-bec782223706 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:23.445330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092361-2641-4c32-a283-bec782223706 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:23.445608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61092361-2641-4c32-a283-bec782223706 None admin] PATH_INFO: `/v3/users/79edd06616304031af478e76fd08fef1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:23.446815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61092361-2641-4c32-a283-bec782223706 None admin] RBAC: Authorizing `identity:get_user(user_id=79edd06616304031af478e76fd08fef1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:23.448245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61092361-2641-4c32-a283-bec782223706 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:23.449859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 553/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/users/79edd06616304031af478e76fd08fef1 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:23.455677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.496311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:23.552451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37108e7-2ef2-4d47-a917-b4a960bb71c5 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:23.587802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:23.588442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:23.588726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:23.589005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:23.653499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:23.654966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:23.693163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:23.749509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 551/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 14:25:23.756774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61092361-2641-4c32-a283-bec782223706 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.784904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:23.854345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61092361-2641-4c32-a283-bec782223706 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:23.887155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:23.887997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:23.888273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:23.888543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:23.889702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:23.891155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cee5b-4d83-4841-af2e-977963dfb8c2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:23.892856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 554/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:23.898848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:23.994559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f7862-7cba-4877-a5da-a23c3fbdd97a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:24.026913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:24.027548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:24.027828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:24.028104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:24.028919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=c8b89bfd17254438a8f8944a30c704bb, user.id=79edd06616304031af478e76fd08fef1) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:24.029230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:24.030685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db9a9f-fb5a-4547-a01c-09fde2e675e8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:24.036462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 552/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:23 2022] GET /identity/v3/role_assignments?user.id=79edd06616304031af478e76fd08fef1&scope.project.id=c8b89bfd17254438a8f8944a30c704bb&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:26.075230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93ddd-3e39-4a3d-badc-97522bb11a43 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:26.075752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93ddd-3e39-4a3d-badc-97522bb11a43 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:26.075752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93ddd-3e39-4a3d-badc-97522bb11a43 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:26.077122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 555/1107] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:26.082894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:26.083341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:26.083529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:26.089347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:26.100312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:26.101495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:26.166428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 553/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:26.191503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a93ddd-3e39-4a3d-badc-97522bb11a43 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:26.283275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:26.290277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a93ddd-3e39-4a3d-badc-97522bb11a43 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:26.322674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:26.323451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:26.323732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:26.324005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:26.325166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:26.326672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba357ab0-5247-4b24-b80d-17c567642558 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:26.328217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 556/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:26.334468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:26.438473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea02b3a-4089-43a3-a32c-7eaa4bf596fa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:26.471401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:26.472061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:26.472344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:26.472647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:26.473442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:26.474893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d49a-a483-4f28-9675-3a1a111d14a0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:26.481280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:26.565160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 554/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:25:26 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:28.629670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64314f-34f0-49f5-86d2-c90878cdeee7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:28.630215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64314f-34f0-49f5-86d2-c90878cdeee7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:28.630215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd64314f-34f0-49f5-86d2-c90878cdeee7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:28.631531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 557/1111] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:28.636723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:28.637142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:28.637274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:28.655896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:28.657077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:28.721018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 555/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:28.744543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64314f-34f0-49f5-86d2-c90878cdeee7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:28.847956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd64314f-34f0-49f5-86d2-c90878cdeee7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:28.881247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:28.882027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:28.882307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:28.882609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] PATH_INFO: `/v3/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:28.884067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] RBAC: Authorizing `identity:get_role(role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:28.885440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:28.886841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 558/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:28 2022] GET /identity/v3/roles/a2ae414a10a54833a1002750699b5d83 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:28.892723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:28.987813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0080a-5914-4781-9ffa-fb75d126c161 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:29.008286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:29.021718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:29.022605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:29.022895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:29.023169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] PATH_INFO: `/v3/users/5bd25954302f43ae90f57e09e3ec8171` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:29.035825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] RBAC: Authorizing `identity:get_user(user_id=5bd25954302f43ae90f57e09e3ec8171)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:29.037254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:29.038965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 556/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:28 2022] GET /identity/v3/users/5bd25954302f43ae90f57e09e3ec8171 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:29.044697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:29.076333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:29.141503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79918e-41ea-4ab4-bbad-9638efeca037 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:29.174686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:29.175322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:29.176197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:29.176476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:29.240268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:29.241721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:29.269837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:29.335982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 559/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:29.341798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:29.438964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e673e-3e7f-447d-ba87-fbee87de6f1d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:29.471050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:29.471893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:29.472172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:29.472957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:29.474127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:29.475589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad4801-953c-473e-af67-a70a49607912 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:29.477289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 557/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:29 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:29.483427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:29.579542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067466bf-3fdd-4e10-8242-4d5f5137bb3b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:29.607536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:29.612186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:29.612822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:29.613701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:29.613972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:29.614812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=c8b89bfd17254438a8f8944a30c704bb, user.id=5bd25954302f43ae90f57e09e3ec8171) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:29.615122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:29.616543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63861e-cd03-4ed9-bd39-3957cc3bc20f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:29.621835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 560/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:29 2022] GET /identity/v3/role_assignments?user.id=5bd25954302f43ae90f57e09e3ec8171&scope.project.id=c8b89bfd17254438a8f8944a30c704bb&role.id=a2ae414a10a54833a1002750699b5d83 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:31.627627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819be6-1b7d-4757-808a-260955b600a3 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:31.628142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819be6-1b7d-4757-808a-260955b600a3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:31.628142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819be6-1b7d-4757-808a-260955b600a3 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:31.630100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 558/1118] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:31.635362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:31.635782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:31.635921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:31.653000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:31.654173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:31.719411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 561/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:31.741872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60819be6-1b7d-4757-808a-260955b600a3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:31.789064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:31.840991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60819be6-1b7d-4757-808a-260955b600a3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:31.872792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:31.873625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:31.873902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:31.874173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] PATH_INFO: `/v3/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:31.875648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] RBAC: Authorizing `identity:get_role(role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:31.877012 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:31.878955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 559/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:31 2022] GET /identity/v3/roles/a2ae414a10a54833a1002750699b5d83 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:31.885259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:31.981842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10169f4e-5e6f-4ec4-bd99-a479807d6d27 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:32.007297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:32.015554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:32.018729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:32.018729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:32.018729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] PATH_INFO: `/v3/users/5bd25954302f43ae90f57e09e3ec8171` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:32.018729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd25954302f43ae90f57e09e3ec8171)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:32.019616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:32.021202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 562/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:31 2022] GET /identity/v3/users/5bd25954302f43ae90f57e09e3ec8171 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:32.027074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:32.121917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e502ce59-28eb-47af-a02f-a388464b591c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:32.136316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:32.155107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:32.155939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:32.156215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:32.156483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:32.157629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:32.159122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd78c09-bb84-4a33-bad7-ea79531d98ce None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:32.160875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 560/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:32 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:32.166677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:32.262075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6a3d2-9c81-4600-8304-832f5d28ee37 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:32.294852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:32.295714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:32.296255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:32.296525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb/users/5bd25954302f43ae90f57e09e3ec8171/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:32.298700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] RBAC: Authorizing `identity:create_grant(project_id=c8b89bfd17254438a8f8944a30c704bb, user_id=5bd25954302f43ae90f57e09e3ec8171, role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:32.300135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d619cc-674f-445a-a3aa-09c29b76bc8b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:32.385012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 563/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:25:32 2022] PUT /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb/users/5bd25954302f43ae90f57e09e3ec8171/roles/a2ae414a10a54833a1002750699b5d83 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:25:34.371934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b4e5-97c7-4b16-9123-d090a363878e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:34.372536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b4e5-97c7-4b16-9123-d090a363878e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:34.372810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6b4e5-97c7-4b16-9123-d090a363878e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:34.374192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 561/1124] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:34.379488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:34.379927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:34.380022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:34.397858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:34.399170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:34.448564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:34.466315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 564/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 14:25:34.491335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6b4e5-97c7-4b16-9123-d090a363878e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:34.591069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6b4e5-97c7-4b16-9123-d090a363878e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:34.623851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:34.624634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:34.624918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:34.625198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] PATH_INFO: `/v3/roles/a2ae414a10a54833a1002750699b5d83` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:34.626697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] RBAC: Authorizing `identity:get_role(role_id=a2ae414a10a54833a1002750699b5d83)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:34.628576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:34.630064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 562/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:34 2022] GET /identity/v3/roles/a2ae414a10a54833a1002750699b5d83 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:34.635830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:34.731662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c77493-a360-4476-958a-aeeaac851f63 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:34.764758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:34.765614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:34.765896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:34.766171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] PATH_INFO: `/v3/users/5bd25954302f43ae90f57e09e3ec8171` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:34.767389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd25954302f43ae90f57e09e3ec8171)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:34.769462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:34.771110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 565/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:34 2022] GET /identity/v3/users/5bd25954302f43ae90f57e09e3ec8171 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:34.776783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:34.826759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:34.874504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31422068-0e99-49c2-91a3-ef6df142c54f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:34.907868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:34.908504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:34.908786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:34.909060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:34.973172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:34.974636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:35.020847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:35.070859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 563/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:35.076696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:35.127303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:35.178610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e3df1-ccef-446b-ae87-45e530b20314 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:35.210540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:35.211387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:35.211665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:35.211934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] PATH_INFO: `/v3/projects/c8b89bfd17254438a8f8944a30c704bb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:35.213079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] RBAC: Authorizing `identity:get_project(project_id=c8b89bfd17254438a8f8944a30c704bb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:35.214497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4048d-deff-4422-bb1f-44aac3be1463 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:35.216181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 566/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:35 2022] GET /identity/v3/projects/c8b89bfd17254438a8f8944a30c704bb => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:35.222359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:35.317922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31668a73-7080-4bf5-b11d-42844e91ca2b None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:35.350583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:35.351222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:35.351505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:35.351779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:35.352592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] RBAC: Adding query filter params (role.id=a2ae414a10a54833a1002750699b5d83, scope.project.id=c8b89bfd17254438a8f8944a30c704bb, user.id=5bd25954302f43ae90f57e09e3ec8171) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:35.352911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:35.354340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4411de-6c3a-4c54-8867-ca48f848b4dc None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:35.360242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 564/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:35 2022] GET /identity/v3/role_assignments?user.id=5bd25954302f43ae90f57e09e3ec8171&scope.project.id=c8b89bfd17254438a8f8944a30c704bb&role.id=a2ae414a10a54833a1002750699b5d83 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:37.406961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113adb-64c2-457b-90ae-16127d6a317d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:37.407450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113adb-64c2-457b-90ae-16127d6a317d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:37.407551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73113adb-64c2-457b-90ae-16127d6a317d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:37.408888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 567/1131] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:37.414109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:37.414492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:37.414709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:37.432722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:37.433921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:37.444478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:37.499752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 565/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:37.524876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113adb-64c2-457b-90ae-16127d6a317d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:37.622925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73113adb-64c2-457b-90ae-16127d6a317d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:37.641910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:37.655052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:37.657541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:37.657541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:37.657541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:37.657541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:37.658962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf882b9-50cf-4eea-b7a5-5d5420ecf222 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:37.661063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 568/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:37.666880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:37.763366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0f15a-e9b3-4302-b8b6-669820446a96 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:37.801430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:37.802075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:37.802352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:37.802654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:37.803357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:37.804786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dd636-1e0b-4545-841f-a562df9ac6c4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:37.818095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:37.902665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 566/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:25:37 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:39.957894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc273be-8265-47ff-844c-4bfa363f633a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:39.958441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc273be-8265-47ff-844c-4bfa363f633a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:39.958588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc273be-8265-47ff-844c-4bfa363f633a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:39.960000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 569/1135] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:39.965164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:39.965590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:39.965696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:39.981886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:39.983093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:40.046065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 567/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:40.071506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc273be-8265-47ff-844c-4bfa363f633a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:40.171626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc273be-8265-47ff-844c-4bfa363f633a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:40.204546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:40.205323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:40.205599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:40.206625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:40.207837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:40.209300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472921e-ed88-4562-9b5b-f3dd67264aba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:40.210881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 570/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:25:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:40.216612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:40.313358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a0876-3c00-46bf-a614-3331a42ca2fc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:40.338102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:40.346324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:40.347581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:40.347867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:40.348144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:40.348853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:40.350283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330b6e-576e-46c8-928c-84dad42a4446 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:40.459116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 568/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:25:40 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:25:42.506310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f246f6-9218-47fe-8eea-04c96df62e8a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:42.506846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f246f6-9218-47fe-8eea-04c96df62e8a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:42.506935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f246f6-9218-47fe-8eea-04c96df62e8a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:42.508274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 571/1139] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:42.513327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757140-c6cd-476c-808f-97256e615800 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:42.513758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757140-c6cd-476c-808f-97256e615800 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:42.513892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e757140-c6cd-476c-808f-97256e615800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:42.531037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0e757140-c6cd-476c-808f-97256e615800 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:42.532233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0e757140-c6cd-476c-808f-97256e615800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:42.597987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 569/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:42.621018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f246f6-9218-47fe-8eea-04c96df62e8a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:42.676454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:42.718899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f246f6-9218-47fe-8eea-04c96df62e8a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:42.751888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:42.752793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:42.753084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:42.753362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:42.763429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:42.771549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:42.772565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:42.773565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 572/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:42.778571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e757140-c6cd-476c-808f-97256e615800 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:42.875228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e757140-c6cd-476c-808f-97256e615800 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:42.888549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:42.909335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:42.909990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:42.910275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:42.910577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:42.911327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:42.911634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:42.913002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:42.921271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 570/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:42.926961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:43.023780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7676aef-0160-441e-99e6-b7cb39aaf700 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:43.039133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:43.056160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:43.057066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:43.057342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:43.057616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] PATH_INFO: `/v3/users/80fca6ca9c724d5aaab1de56b291fa1c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:43.071279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] RBAC: Authorizing `identity:get_user(user_id=80fca6ca9c724d5aaab1de56b291fa1c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:43.072740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:43.074379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 573/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:42 2022] GET /identity/v3/users/80fca6ca9c724d5aaab1de56b291fa1c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:43.080087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:43.177358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12476c-2dfa-41d6-8935-47b13e13f425 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:43.210386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:43.211044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:43.211326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:43.211601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:43.253580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:43.277755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:43.279232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:43.375710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 571/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:43.382151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:43.477883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf661a-6e6a-4a45-a5c2-b39014ac8c98 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:43.510731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:43.511566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:43.511845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:43.512115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] PATH_INFO: `/v3/projects/b90b390ee25e440fa75796b5f2aa5125` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:43.513268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] RBAC: Authorizing `identity:get_project(project_id=b90b390ee25e440fa75796b5f2aa5125)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:43.514720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21599a-5909-485a-95dc-539a724c1372 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:43.516380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 574/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:43 2022] GET /identity/v3/projects/b90b390ee25e440fa75796b5f2aa5125 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:43.522271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:43.593495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:43.617730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb891d04-c6b3-47aa-91f4-6db06ee6f9cb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:43.650427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:43.651097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:43.651407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:43.651684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:43.652498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=b90b390ee25e440fa75796b5f2aa5125, user.id=80fca6ca9c724d5aaab1de56b291fa1c) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:43.652811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:43.654233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb8007-d236-400b-8a77-b253f57ad522 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:43.660249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 572/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:43 2022] GET /identity/v3/role_assignments?user.id=80fca6ca9c724d5aaab1de56b291fa1c&scope.project.id=b90b390ee25e440fa75796b5f2aa5125&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:45.620252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176fa62-0c8d-4a2e-8b18-b792ef3da68c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:45.620773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176fa62-0c8d-4a2e-8b18-b792ef3da68c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:45.620773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176fa62-0c8d-4a2e-8b18-b792ef3da68c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:45.622112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 575/1147] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:45.627867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:45.628267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:45.628517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:45.645074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:45.646239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:45.709723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 573/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:45.731532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176fa62-0c8d-4a2e-8b18-b792ef3da68c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:45.775418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:45.830700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176fa62-0c8d-4a2e-8b18-b792ef3da68c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:45.863897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:45.864658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:45.864935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:45.865206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:45.875004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:45.882686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:45.883648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:45.884538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 576/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:25:45.889836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:45.953629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:45.985814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec93df3-626b-43f9-944a-0c8b0159092b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:46.018790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:46.019449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:46.019730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:46.020008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:46.020749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:46.021057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:46.022426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:46.030685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 574/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:46.036616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:46.133014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768ad95-a760-4dab-a120-296b8db72244 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:46.134469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:46.166456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:46.167343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:46.167622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:46.167896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] PATH_INFO: `/v3/users/80fca6ca9c724d5aaab1de56b291fa1c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:46.169057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=80fca6ca9c724d5aaab1de56b291fa1c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:46.170831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:46.172434 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 577/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:46 2022] GET /identity/v3/users/80fca6ca9c724d5aaab1de56b291fa1c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:46.178730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:46.275225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1040af-94f6-4262-beef-0f8f4f201550 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:46.302725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:46.308225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:46.309120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:46.309398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:46.314398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] PATH_INFO: `/v3/projects/b90b390ee25e440fa75796b5f2aa5125` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:46.314398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] RBAC: Authorizing `identity:get_project(project_id=b90b390ee25e440fa75796b5f2aa5125)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:46.314398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07554ec-dda8-4d3c-bff1-1011699e0013 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:46.314398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 575/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:46 2022] GET /identity/v3/projects/b90b390ee25e440fa75796b5f2aa5125 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:46.319927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:46.415666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff961d-0487-4cb4-b93c-9f55f51e7cc0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:46.448147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:46.449009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:46.449285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:46.449551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] PATH_INFO: `/v3/projects/b90b390ee25e440fa75796b5f2aa5125/users/80fca6ca9c724d5aaab1de56b291fa1c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:46.451496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] RBAC: Authorizing `identity:create_grant(project_id=b90b390ee25e440fa75796b5f2aa5125, user_id=80fca6ca9c724d5aaab1de56b291fa1c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:46.452909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c8114-282d-4836-b6fc-2fe687f14317 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:46.539162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 578/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:25:46 2022] PUT /identity/v3/projects/b90b390ee25e440fa75796b5f2aa5125/users/80fca6ca9c724d5aaab1de56b291fa1c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:25:48.541847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f6f4-4bfe-4c7c-9ff8-a359562df9a2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:48.542383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f6f4-4bfe-4c7c-9ff8-a359562df9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:48.542474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f6f4-4bfe-4c7c-9ff8-a359562df9a2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:48.543993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 576/1154] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:48.549329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:48.549747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:48.549882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:48.566295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:48.567508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:48.591579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:48.633226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 579/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:48.657441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f6f4-4bfe-4c7c-9ff8-a359562df9a2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:48.756699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f6f4-4bfe-4c7c-9ff8-a359562df9a2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:48.789500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:48.790268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:48.790571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:48.790849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:48.801441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:25:48.808913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:48.810658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:25:48.811511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 577/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:48.816149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:48.911974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379bf0e4-90b6-4b2b-bd4b-261e1dbff4e6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:48.946551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:48.947206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:48.947488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:48.947763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:48.948498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:48.948803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:48.950170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:48.958821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 580/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:48.964759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:49.021438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:49.061395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402e3c4-f483-409d-9e16-4b51bc90da99 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:49.093601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:49.094447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:49.094884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:49.095162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] PATH_INFO: `/v3/users/80fca6ca9c724d5aaab1de56b291fa1c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:49.096942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] RBAC: Authorizing `identity:get_user(user_id=80fca6ca9c724d5aaab1de56b291fa1c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:49.098367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:49.100008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 578/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:25:48 2022] GET /identity/v3/users/80fca6ca9c724d5aaab1de56b291fa1c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:49.105738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:49.131506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:49.206425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988f176-62f8-4851-a42c-87065fd04350 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:49.239316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:49.239961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:49.240244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:49.240520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:49.306710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:49.308164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:49.331325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:49.406459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 581/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:49.412991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:49.509603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec29b13-a26c-49d1-b2ab-f9e017c2adfb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:49.535717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:49.544042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:49.547484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:49.547484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:49.547484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] PATH_INFO: `/v3/projects/b90b390ee25e440fa75796b5f2aa5125` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:49.547484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] RBAC: Authorizing `identity:get_project(project_id=b90b390ee25e440fa75796b5f2aa5125)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:49.548277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68eab7-d3d0-40ac-aa2c-ef1da22d021c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:49.550088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 579/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:25:49 2022] GET /identity/v3/projects/b90b390ee25e440fa75796b5f2aa5125 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:49.556278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:49.652595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77264f5d-9f6e-46d0-819a-4d48be2a5019 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:49.673694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:49.684627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:49.685248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:49.685523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:49.685791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:49.686625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=b90b390ee25e440fa75796b5f2aa5125, user.id=80fca6ca9c724d5aaab1de56b291fa1c) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:49.686936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:49.688353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a48250-76ec-4cf3-b8e1-310204150db7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:49.694583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 582/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:25:49 2022] GET /identity/v3/role_assignments?user.id=80fca6ca9c724d5aaab1de56b291fa1c&scope.project.id=b90b390ee25e440fa75796b5f2aa5125&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:51.812423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765bae-6336-464c-8d2d-649070c74c1f None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:51.812953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765bae-6336-464c-8d2d-649070c74c1f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:51.812953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b765bae-6336-464c-8d2d-649070c74c1f None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:51.815430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 580/1162] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:51.820637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:51.821005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:51.821183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:51.837663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:51.838879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:51.903397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 583/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:51.926816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765bae-6336-464c-8d2d-649070c74c1f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:52.023431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b765bae-6336-464c-8d2d-649070c74c1f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:52.056933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:52.057564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:52.057847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:52.058123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:52.116200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:52.123192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:52.124627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:52.218705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 581/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:25:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:25:52.225427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:52.319715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b965d-5c87-4e15-bed5-9b7f2904b581 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:52.335814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:52.353625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:52.354417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:52.354720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:52.354990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:52.362040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:52.363500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-97f83a3a-7263-4147-9968-d7240dcdd186 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:25:52.378023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:25:52.379291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:25:52.380358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 584/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:25:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:25:52.385083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:52.474873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:52.481193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87646548-01c6-4f59-b02e-7191ace6a017 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:52.514579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:52.515216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:52.515495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:52.515765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:52.516467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:25:52.516767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:52.518677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2fdfb-317f-491b-8ecf-a17cb6018d0c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:52.525499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 582/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Fri Jun 24 14:25:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:54.533072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874a49-002b-42be-9c51-c37a89863b78 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:54.533589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874a49-002b-42be-9c51-c37a89863b78 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:54.533589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13874a49-002b-42be-9c51-c37a89863b78 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:54.535136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 585/1167] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:54.540246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2ff-f5b5-4862-aab5-d35da4865e2c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:54.540645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2ff-f5b5-4862-aab5-d35da4865e2c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:54.540803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25d2ff-f5b5-4862-aab5-d35da4865e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:54.558377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ee25d2ff-f5b5-4862-aab5-d35da4865e2c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:54.559666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee25d2ff-f5b5-4862-aab5-d35da4865e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:54.625476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 583/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:54.649488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874a49-002b-42be-9c51-c37a89863b78 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:54.747327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13874a49-002b-42be-9c51-c37a89863b78 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:54.779386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:54.780021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:54.780297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:54.780568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:54.781286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:54.782777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae56c4-7cba-4fa3-baca-46c58c4bd249 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:54.844701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 586/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Fri Jun 24 14:25:54 2022] POST /identity/v3/domains => generated 267 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:56.861449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885bf8c-cf7e-452f-a90c-8da4e37922f7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:56.861992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885bf8c-cf7e-452f-a90c-8da4e37922f7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:56.861992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885bf8c-cf7e-452f-a90c-8da4e37922f7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:56.863387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 584/1170] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:56.868622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:56.869048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:56.869178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:56.886085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:56.887301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:56.919506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:56.957315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 587/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:56.980619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885bf8c-cf7e-452f-a90c-8da4e37922f7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:57.067696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:57.081082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885bf8c-cf7e-452f-a90c-8da4e37922f7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:57.114981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:57.115765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:57.116047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:57.116320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] PATH_INFO: `/v3/domains/9cf861e348ff4fde8b3dbdd4eb04f2a8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:57.126383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=9cf861e348ff4fde8b3dbdd4eb04f2a8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:57.127862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b8a7-8820-40c3-9449-15fed9c83eba None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:57.129488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 585/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Jun 24 14:25:56 2022] GET /identity/v3/domains/9cf861e348ff4fde8b3dbdd4eb04f2a8 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:57.135316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:57.229542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef9e1f-7870-4400-9193-96b95aeab36f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:57.262150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:57.262821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:57.263104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:57.263379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:57.264082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:57.265513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5f83aa-4fa6-4f94-bf84-4687a6df4591 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:57.367161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 588/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:25:57 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:25:59.412492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00303d-0f56-4ff1-8b69-719612f2068c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:59.412997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00303d-0f56-4ff1-8b69-719612f2068c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:59.413097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00303d-0f56-4ff1-8b69-719612f2068c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:59.414461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 586/1174] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:25:59.419785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:59.420180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:59.420343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:59.437927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:25:59.439352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:25:59.452095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:25:59.504680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 589/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:25:59.529173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00303d-0f56-4ff1-8b69-719612f2068c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:59.632078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00303d-0f56-4ff1-8b69-719612f2068c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:59.665325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:59.666167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:59.666472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:59.666785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] PATH_INFO: `/v3/domains/9cf861e348ff4fde8b3dbdd4eb04f2a8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:59.667968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] RBAC: Authorizing `identity:get_domain(domain_id=9cf861e348ff4fde8b3dbdd4eb04f2a8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:59.669668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfab4e5-db26-4c96-9ab8-88e2766831df None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:59.671318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 587/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Fri Jun 24 14:25:59 2022] GET /identity/v3/domains/9cf861e348ff4fde8b3dbdd4eb04f2a8 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:25:59.677781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:25:59.773852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25889a69-d567-4c64-8633-21acb9a59dfe None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:25:59.807097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:25:59.807752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:25:59.808035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:25:59.808312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:25:59.809005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:25:59.810436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbe158-69e8-4c9e-bde0-b7dd915d408e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:25:59.902034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 590/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:25:59 2022] POST /identity/v3/users => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:26:01.973762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ef46-d0ed-4ca9-90c7-dbb4ddc928a6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:01.974318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ef46-d0ed-4ca9-90c7-dbb4ddc928a6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:01.974318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9ef46-d0ed-4ca9-90c7-dbb4ddc928a6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:01.975704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 588/1178] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:26:01.980812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:01.981203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:01.981365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:01.987502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:01.997856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:26:02.000911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:26:02.063916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 591/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:26:02.088033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9ef46-d0ed-4ca9-90c7-dbb4ddc928a6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:02.106581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:02.190680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9ef46-d0ed-4ca9-90c7-dbb4ddc928a6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:02.224041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:02.224748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:02.225025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:02.225295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:02.235602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:26:02.243139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:26:02.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:26:02.245498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 589/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:26:02.250501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:02.345293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3646e5-48b2-488c-80ee-2ea06703a64f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:02.378019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:02.378710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:02.378990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:02.379258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:02.379990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:26:02.380293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:02.381647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:02.382987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:02.389970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 592/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:02.399862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:02.470421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:02.491987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17cfbe8-9618-4af6-9534-9ab0b3d38475 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:02.525471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:02.526317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:02.526622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:02.526895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] PATH_INFO: `/v3/users/320fe56a6f2d4fe9b7539c781f3724d1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:02.540521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] RBAC: Authorizing `identity:get_user(user_id=320fe56a6f2d4fe9b7539c781f3724d1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:02.541951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:02.543851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 590/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/users/320fe56a6f2d4fe9b7539c781f3724d1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:02.549755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:02.645699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f940884-d2b6-4c7b-a7a8-666876ead1e5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:02.677042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:02.677668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:02.677945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:02.678216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:02.736048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:02.742256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:02.743676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:02.839755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 593/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:26:02.845691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:02.946083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661ee34-fc64-4df5-98ac-8bc19a250cea None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:02.978852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:02.979698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:02.979981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:02.980260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] PATH_INFO: `/v3/projects/cd5665e245ab49b7b844059f0618606a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:02.981422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] RBAC: Authorizing `identity:get_project(project_id=cd5665e245ab49b7b844059f0618606a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:02.982879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9951dec-5d16-424d-a210-3e9bde0230d5 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:02.984581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 591/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/projects/cd5665e245ab49b7b844059f0618606a => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:26:02.991329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:03.083301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:03.088829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf6dc0-c60d-45ae-bf3d-58e1f93c07cf None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:03.122116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:03.122792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:03.123071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:03.123345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:03.124153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=cd5665e245ab49b7b844059f0618606a, user.id=320fe56a6f2d4fe9b7539c781f3724d1) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:26:03.124458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:03.125909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e204b99-f83d-42a6-8c15-47b702ba8ff9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:03.131458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 594/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:26:02 2022] GET /identity/v3/role_assignments?user.id=320fe56a6f2d4fe9b7539c781f3724d1&scope.project.id=cd5665e245ab49b7b844059f0618606a&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:05.141053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd3f3-c5ed-4978-95fe-bd7a23fe9e5e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.141614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd3f3-c5ed-4978-95fe-bd7a23fe9e5e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.141614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225dd3f3-c5ed-4978-95fe-bd7a23fe9e5e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.143093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 592/1186] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:26:05.148181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.148592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.148738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.165547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:26:05.166765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:26:05.233633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 595/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:26:05.257124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dd3f3-c5ed-4978-95fe-bd7a23fe9e5e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:05.259982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:05.355518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225dd3f3-c5ed-4978-95fe-bd7a23fe9e5e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:05.388376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:05.389150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.389429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.389698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.399534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:26:05.407580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:26:05.408712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:26:05.409671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 593/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:26:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:26:05.414965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:05.501910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:05.510773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf850d-9b24-499d-afbf-e1f2f1ed19cc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:05.544198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:05.544901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.545185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.545461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.546212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:26:05.546549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:05.548746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:05.557365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 596/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:26:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:05.564118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:05.627259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:05.664709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e19fe-8157-4b86-8265-86154b16c1ef None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:05.696898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:05.697753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.698033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.698305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] PATH_INFO: `/v3/users/320fe56a6f2d4fe9b7539c781f3724d1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.699510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] RBAC: Authorizing `identity:get_user(user_id=320fe56a6f2d4fe9b7539c781f3724d1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:05.700935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:05.702566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 594/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:26:05 2022] GET /identity/v3/users/320fe56a6f2d4fe9b7539c781f3724d1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:05.709010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:05.805286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024038c-9734-4ee4-97d7-0c6427cfc710 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:05.837280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:05.838116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.838393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.838690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] PATH_INFO: `/v3/projects/cd5665e245ab49b7b844059f0618606a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.839850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] RBAC: Authorizing `identity:get_project(project_id=cd5665e245ab49b7b844059f0618606a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:05.841259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2f0d9-30b4-4776-a972-144655204378 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:05.842942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 597/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:26:05 2022] GET /identity/v3/projects/cd5665e245ab49b7b844059f0618606a => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:05.849379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:05.944320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56121aa6-1e19-4709-9307-c9e149183003 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:05.971646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:05.977148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:05.978052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:05.980585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:05.980585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] PATH_INFO: `/v3/projects/cd5665e245ab49b7b844059f0618606a/users/320fe56a6f2d4fe9b7539c781f3724d1/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:05.980585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd5665e245ab49b7b844059f0618606a, user_id=320fe56a6f2d4fe9b7539c781f3724d1, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:05.982085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f827d99-e09f-41c9-a716-86103e21dbf2 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:06.067071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 595/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:26:05 2022] PUT /identity/v3/projects/cd5665e245ab49b7b844059f0618606a/users/320fe56a6f2d4fe9b7539c781f3724d1/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:26:08.055631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed1cf6-f02d-4ea1-9694-d2fd502357b2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.056154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed1cf6-f02d-4ea1-9694-d2fd502357b2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.056154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed1cf6-f02d-4ea1-9694-d2fd502357b2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.057543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 598/1193] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:26:08.062723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.063116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.063372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.080490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:26:08.081658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:26:08.146248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 596/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:26:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:26:08.169674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed1cf6-f02d-4ea1-9694-d2fd502357b2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:08.192676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:08.269199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed1cf6-f02d-4ea1-9694-d2fd502357b2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:08.302072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:08.302875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.303164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.303440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.314101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:26:08.321729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:26:08.322834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:26:08.323895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 599/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:26:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:26:08.329090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:08.427078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a25eb-11ba-4d4d-b016-dd05dd0b200b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:08.446853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:08.460486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:08.461152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.461435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.461711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.462454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:26:08.462791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:08.464152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:08.472650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 597/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:26:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:08.478468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:08.562377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:08.581817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fd37f-82bf-41e5-bec2-83f6b679be98 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:08.614532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:08.615395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.615674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.615946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] PATH_INFO: `/v3/users/320fe56a6f2d4fe9b7539c781f3724d1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.617123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] RBAC: Authorizing `identity:get_user(user_id=320fe56a6f2d4fe9b7539c781f3724d1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:08.618604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:08.620245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 600/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Fri Jun 24 14:26:08 2022] GET /identity/v3/users/320fe56a6f2d4fe9b7539c781f3724d1 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:08.626644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:08.724574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f622c-511d-43c2-bdf1-5d7ee32c9f23 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:08.756338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:08.756979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:08.757260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:08.757533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:08.804913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:08.822400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:08.823884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:08.919468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 598/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:26:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:26:08.925227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:09.020068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea51ce4-a994-4300-a203-6bbc2f50c578 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:09.052455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:09.053300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:09.053577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:09.053846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] PATH_INFO: `/v3/projects/cd5665e245ab49b7b844059f0618606a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:09.055012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] RBAC: Authorizing `identity:get_project(project_id=cd5665e245ab49b7b844059f0618606a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:09.056424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e630f-6b28-46dd-ad47-affc8fc8daa0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:09.058140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 601/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Fri Jun 24 14:26:08 2022] GET /identity/v3/projects/cd5665e245ab49b7b844059f0618606a => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:09.064248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:26:09.143423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:09.161068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e31ddb-2dda-4780-8ff8-f5f8ecc454c3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:26:09.199568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:26:09.200156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:26:09.200439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:26:09.200714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:26:09.202975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=cd5665e245ab49b7b844059f0618606a, user.id=320fe56a6f2d4fe9b7539c781f3724d1) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:26:09.202975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:26:09.204224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c862b2a1-d698-4947-a466-9db1b60c0e66 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:26:09.210186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 599/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:26:09 2022] GET /identity/v3/role_assignments?user.id=320fe56a6f2d4fe9b7539c781f3724d1&scope.project.id=cd5665e245ab49b7b844059f0618606a&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:26:19.048467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:26:19.207488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=27 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:21.364299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bda1-358c-44d1-8850-5217e5221468 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:21.364896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bda1-358c-44d1-8850-5217e5221468 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:21.364896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430bda1-358c-44d1-8850-5217e5221468 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:21.366695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 602/1201] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:21.371321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:21.373034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:21.373034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:21.391081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:21.392259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:21.461451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 600/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:21.485829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430bda1-358c-44d1-8850-5217e5221468 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:21.585415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430bda1-358c-44d1-8850-5217e5221468 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:21.618753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:21.618753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:21.618753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:21.618753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:21.629487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:21.631089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:21.648837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:21.649987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:21.651243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:21.651243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:21.651243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:21.651243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:21.651243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 603/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:21.656187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:21.757963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd98ff-9afe-424a-873b-bcc976bfc962 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:21.787893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:21.793885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:21.794822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:21.795690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:21.796063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:21.796983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:21.797299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:21.798712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc6834-687f-4b73-83f1-12ce08ef3d8f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:21.810400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 601/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:21.816726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:21.868608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:21.914297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce231b7c-907a-4be3-8653-61b498154c1f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:21.947206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:21.947871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:21.952404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:21.952404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:21.952404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:21.952404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e657d-e1ad-4f2d-be85-f706f6c6c276 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:22.015202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 604/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Fri Jun 24 14:27:21 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:27:24.062221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8391-ac8c-4395-ae8a-243dcfd353ff None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.062805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8391-ac8c-4395-ae8a-243dcfd353ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.062805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae8391-ac8c-4395-ae8a-243dcfd353ff None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.064311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 602/1206] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:24.069558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.070017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.070112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.087196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:24.088380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:24.152706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 605/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:24.176873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae8391-ac8c-4395-ae8a-243dcfd353ff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:24.280187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae8391-ac8c-4395-ae8a-243dcfd353ff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:24.313034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:24.313757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.314045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.314320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.324760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:24.333084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:24.334214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:24.335262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 603/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:24.340495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:24.381232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:24.439132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcda8e-fbef-490d-9533-e6aafd13df32 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:24.471996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:24.472654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.472931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.473201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.473934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:24.474233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:24.475623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:24.483378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 606/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:24.489110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:24.540503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:24.589110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385c6cb-762f-4f1a-b5f3-9e87df20c0cb None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:24.627260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:24.628076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.628345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.628623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.638505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:24.641112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:24.658898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:24.659909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:24.661011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:24.661011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:24.661011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:24.661011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:24.661821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 604/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:24.666640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:24.761181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:24.764517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6ecf3-01eb-4650-8b9b-cba54e5f481e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:24.797297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:24.797941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.798219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.798490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.799223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:24.799528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:24.800883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:24.812251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 607/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:24.818052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:24.916038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94542dd0-e3b0-4677-8936-b1e92a91e221 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:24.917518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:24.950278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:24.951131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:24.951413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:24.951699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:24.958494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:24.959956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:24.974379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:24.980515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:24.981693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 605/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:24.983318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:25.081637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a5a6c-49ed-487f-b6fd-18c47ef9e1b2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:25.116165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:25.116834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:25.117114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:25.117386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:25.118214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:25.118552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:25.119989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:25.137237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 608/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:25.142974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:25.240307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e7fd2-23f6-4f3e-acd2-4b46e16c479e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:25.272861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:25.273498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:25.273778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:25.274050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:25.304252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:25.340808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:25.343151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:25.442477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 606/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:27:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:27:25.448539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:25.467525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:25.548221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c404c-1d47-4d8e-9ad6-e2dcb72cec6d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:25.582075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:25.582898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:25.583179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:25.583451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:25.593827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:25.595285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:25.612375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:25.613472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:25.614625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:25.614625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:25.614625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:25.614625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:25.614625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 609/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:25.619531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:25.685160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:25.727695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1ef90-8429-46b7-9de4-015ba803a1b3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:25.760372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:25.761020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:25.761299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:25.761646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:25.762355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:25.762686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:25.764627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:25.775333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 607/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:25.781572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:25.849388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:25.880089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b892d73-6b6d-4d19-94f7-be0ff3f38995 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:25.917546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:25.918333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:25.918640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:25.918916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:25.925582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:25.927058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:25.942357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:25.943360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:25.944448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 610/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:25.949458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:26.048690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced0f0-6fe7-4e25-9f96-958f0d20291a None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:26.070540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:26.082224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:26.082918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:26.083199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:26.083471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:26.084245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:26.084552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:26.085978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f0ed1-2125-4579-a0d7-b7cd5ce82281 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:26.097536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 608/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:26.103760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:26.202186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f44740-2dfa-4e1f-a9a2-4eb9847bd690 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:26.231348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:26.237837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:26.238487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:26.238799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:26.239306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:26.240124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=812d75984e6245e2939c28fbd068b895) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:26.240433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:26.242148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a05167-81f5-407e-afc6-45ce59b87247 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:26.247637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 611/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:27:26 2022] GET /identity/v3/role_assignments?user.id=812d75984e6245e2939c28fbd068b895&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:28.300515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57810-6fa6-4fb5-96fe-e33911df0e18 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:28.301071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57810-6fa6-4fb5-96fe-e33911df0e18 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:28.301071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b57810-6fa6-4fb5-96fe-e33911df0e18 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:28.302455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 609/1220] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:28.307773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:28.308154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:28.308323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:28.324726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:28.325925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:28.393493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 612/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:28.417177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57810-6fa6-4fb5-96fe-e33911df0e18 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:28.519429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b57810-6fa6-4fb5-96fe-e33911df0e18 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:28.553476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:28.554288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:28.554592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:28.554867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:28.564878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:28.572513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:28.578170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:28.579306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 610/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:27:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:28.580501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:28.676758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a6b1d3-2a49-40fe-a023-b1aaa28299fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:28.699743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:28.710496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:28.711183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:28.711472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:28.711742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:28.712477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:28.712785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:28.714153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:28.722784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 613/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:27:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:28.728568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:28.772016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:28.826930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35d532-6a36-45b0-8b62-2cc73d0e84a3 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:28.859181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:28.859958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:28.860241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:28.860519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:28.871599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:28.873711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:28.891039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:28.892055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:28.893268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:28.893268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:28.893268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:28.893268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:28.893268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 611/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:28.898476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:28.996579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d6ecc-221d-4707-836d-2ac2e7288b93 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.029717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.033151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.033151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.033151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.033151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:29.033151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.033580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.045160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 614/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:29.050922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.148350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:29.155506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156867-6fc0-4191-8aeb-3bd39f932863 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.189341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.190155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.190440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.190747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.198303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.200120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:29.215431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:29.216521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:29.222299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 612/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:29.223051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.271304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:29.324777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433a751-40c5-40b0-8cf9-30deafa9e81a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.358157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.358861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.359145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.359418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.360251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:29.360565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.362045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.381696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 615/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:29.387633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.488452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca998ef-4bc2-4db9-bca9-9e8835b3f9d7 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.523084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.523876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.524159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.524431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.534770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.536467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:29.553767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:29.554871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:29.555971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:29.555971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:29.555971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:29.555971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:29.555971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 613/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:29.561658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.664932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b2768-832f-4251-b35a-cbe2a1d85571 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.667198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:29.697557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.698222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.698503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.698809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.699521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:29.699828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.701511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.712351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 616/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:29.718702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.728883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:29.816051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332f142-04fb-4aad-bdd8-6b1199210ea8 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:29.849280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:29.850092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:29.851251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:29.851523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:29.859115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:29.867631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:29.874721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:29.875747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:29.876656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 614/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:29.881788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:29.980680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c4df62-2182-4e59-999e-e8239f3711e4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:30.014408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:30.015106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:30.015391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:30.015665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:30.016451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:30.017040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:30.018204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb64a04-de43-4277-8103-59243a99b03c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:30.031172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 617/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:30.037850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:30.102282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:30.135105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c31b06-b6da-4bb2-b3b7-983a384b9817 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:30.168353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:30.169223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:30.169507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:30.169771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/812d75984e6245e2939c28fbd068b895/roles/c6c8015c5ad64912ae5548d376a938be` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:30.183618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=812d75984e6245e2939c28fbd068b895, role_id=c6c8015c5ad64912ae5548d376a938be)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:30.185084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07009e94-820a-4c0a-9f6b-c98c8602f363 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:30.254182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 615/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:27:30 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/812d75984e6245e2939c28fbd068b895/roles/c6c8015c5ad64912ae5548d376a938be => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:27:32.295035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f95d7c-ce99-4b45-bfad-13c8a727753d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:32.295607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f95d7c-ce99-4b45-bfad-13c8a727753d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:32.295607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f95d7c-ce99-4b45-bfad-13c8a727753d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:32.296953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 618/1233] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:32.301991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:32.302361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:32.302563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:32.318868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:32.320045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:32.386712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 616/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:32.412025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f95d7c-ce99-4b45-bfad-13c8a727753d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:32.436755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:32.510918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f95d7c-ce99-4b45-bfad-13c8a727753d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:32.546253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:32.547022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:32.547296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:32.547564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:32.557717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:32.565615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:32.576912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:32.578057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 619/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Fri Jun 24 14:27:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:32.579083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:32.620209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:32.673054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369abf9e-4119-423c-95c1-e5c4a7244484 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:32.707285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:32.707933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:32.708222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:32.708496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:32.709240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:32.709546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:32.710949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:32.719591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 617/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Fri Jun 24 14:27:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:32.725283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:32.812285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:32.828851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23468350-6530-4ec7-9f7c-33dbda861e3e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:32.862749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:32.863532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:32.863836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:32.864116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:32.874885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:32.876418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:32.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:32.896753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:32.898172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:32.898172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:32.898172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:32.898172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:32.898172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 620/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:32.902681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.004110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455cf02-13e2-4d15-a4c7-44708a1cdd0d None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:33.014550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:33.039032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:33.039689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:33.039974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:33.040253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:33.040997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:33.041307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:33.042700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:33.054807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 618/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:33.061115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.159745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115304a-dae6-4fb4-b58e-64d9e638ae6a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:33.193691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:33.194480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:33.194788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:33.195058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:33.197687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:33.202571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:33.204004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:33.218448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:33.224657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:33.225824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 621/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:33.226623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.324075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a245aa3-e3b5-45b8-a964-ada0285da180 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:33.357054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:33.357748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:33.358130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:33.358411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:33.359271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:33.359584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:33.361071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:33.378893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 619/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:33.384615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.483968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d72f1a-82de-42cb-8917-18aac3bf5a6c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:33.516027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:33.516666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:33.516946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:33.517220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:33.576492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:33.582849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:33.584305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:33.686427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 622/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:27:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:27:33.692344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.720502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:33.796281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b46681d-afe6-422e-970d-bf843ca7ff14 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:33.832850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:33.833639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:33.833926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:33.834207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:33.844886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:33.847019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:33.864309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:33.877351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:33.878579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:33.878579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:33.878579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:33.878579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:33.878579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 620/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:33.878948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:33.964390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:33.970825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa291e-4611-420a-9f3e-d72952e89acc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:34.004228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:34.004920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:34.005207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:34.005477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:34.006226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:34.006554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:34.008056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:34.020004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 623/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:34.025916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:34.104673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:34.125684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5b00e-5c21-4545-9696-9c0fc29c7e8f None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:34.158798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:34.159582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:34.159858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:34.160126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:34.166887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:34.168364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:34.181722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:34.182806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:34.184089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 621/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:34.189347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:34.287783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44b1d36-b00f-4497-95a0-46ff6ecf3986 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:34.321817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:34.322475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:34.322788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:34.323062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:34.323838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:34.324149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:34.325693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44022a9b-f141-4c22-9bed-29b511806153 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:34.338191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 624/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:34.344360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:34.443091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a5e90-0de1-47d3-811a-232947abd67e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:34.476077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:34.476725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:34.477005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:34.477278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:34.478098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] RBAC: Adding query filter params (role.id=c6c8015c5ad64912ae5548d376a938be, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=812d75984e6245e2939c28fbd068b895) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:34.478409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:34.479871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663a1fb-4587-4df9-9fd2-fea5564b3a34 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:34.485839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 622/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:27:34 2022] GET /identity/v3/role_assignments?user.id=812d75984e6245e2939c28fbd068b895&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=c6c8015c5ad64912ae5548d376a938be => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:36.584731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac7c23-e87f-4dce-ab2e-4fa171cd40aa None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:36.585284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac7c23-e87f-4dce-ab2e-4fa171cd40aa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:36.585284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac7c23-e87f-4dce-ab2e-4fa171cd40aa None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:36.586680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 625/1247] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:36.592022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:36.592503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:36.592603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:36.597768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:36.610060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:36.611287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:36.676635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 623/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:36.702336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac7c23-e87f-4dce-ab2e-4fa171cd40aa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:36.720288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:36.801798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac7c23-e87f-4dce-ab2e-4fa171cd40aa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:36.834823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:36.835646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:36.835923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:36.836193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:36.846017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:36.853538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:36.854669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:36.855870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 626/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:27:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:36.861359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:36.958393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2bee-0796-45b1-ac45-bb24fc941655 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:36.988106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:36.993163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:36.993831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:36.994118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:36.994394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:36.995217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:36.995527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:36.997525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.005584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 624/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:27:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:37.011268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:37.090474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:37.112946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753cf82-823f-4fdd-a5b5-685e32e9fad2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:37.146010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:37.146823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:37.147107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:37.147472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:37.159465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:37.161607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:37.179769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:37.180837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:37.181851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:37.181851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:37.181851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:37.181851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:37.181851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 627/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:37.187064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:37.283657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f16178-1d85-462c-be2e-0740a2ac0a02 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:37.316189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:37.316848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:37.317130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:37.317403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:37.318123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:37.318433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:37.319835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.332537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 625/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:37.338123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:37.438011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb98c-aadd-4093-8cb3-4b11fd5eb226 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:37.462921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:37.470871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:37.471660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:37.471939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:37.472210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:37.479420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:37.480854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:37.495559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:37.496649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:37.497913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 628/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:37.502769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:37.557078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:37.600085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b1664-1909-4aca-befd-176891c6a6ea None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:37.634762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:37.635460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:37.635741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:37.636016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:37.636848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:37.637160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:37.639206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.656588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 626/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:37.662726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:37.761203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50d622-7986-4a62-87fc-197551a9e19f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:37.796144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:37.796797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:37.797079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:37.797355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:37.852261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:37.863938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:37.865396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:37.966826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 629/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:27:37.972864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:38.072871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad579d-7ae5-4bcb-ba91-9e71e3dd4726 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:38.085957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:38.109015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:38.109752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:38.110034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:38.110305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:38.120583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:38.122443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:38.139999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:38.141035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:38.142391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:38.142391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:38.142391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:38.142391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:38.142391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 627/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:27:38.146970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:38.233877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:38.245494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f6fdd-3a9d-4b90-a115-c152f4cfcd04 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:38.279536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:38.280197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:38.280529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:38.280804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:38.281521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:38.281830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:38.283234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:38.294798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 630/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:38.300652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:38.399079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e178c-1ed1-4d53-9d8e-dca029de12d9 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:38.432262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:38.433044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:38.433326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:38.433600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:38.440907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:38.442342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:38.456813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:38.458654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:38.459815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 628/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:38.464135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:38.562029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca9eb2-58dd-4289-9bb0-63b13457aab6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:38.595257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:38.595909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:38.596208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:38.596451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:38.597220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:38.597527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:38.598976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd6e6c-ff23-4269-af91-ad7a79633692 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:38.610578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 631/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:38.616781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:38.643268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:38.717740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2967867d-501d-4c98-83ba-1b57b3626619 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:38.750923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:38.751565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:38.751845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:38.752121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:38.752930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=812d75984e6245e2939c28fbd068b895) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:38.753240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:38.754685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e25ef-a689-4451-b7e9-d5bb0d4cbf2b None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:38.760077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 629/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:27:38 2022] GET /identity/v3/role_assignments?user.id=812d75984e6245e2939c28fbd068b895&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:40.754711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd2ce-3a75-4452-988f-12dfa0c138a9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:40.755234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd2ce-3a75-4452-988f-12dfa0c138a9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:40.755234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acd2ce-3a75-4452-988f-12dfa0c138a9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:40.756595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 632/1261] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:40.761635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:40.762700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:40.762844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:40.779875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:40.781050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:40.846908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 630/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:40.869230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd2ce-3a75-4452-988f-12dfa0c138a9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:40.880923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:40.972931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acd2ce-3a75-4452-988f-12dfa0c138a9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.007104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.007829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.008115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.008393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.019449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:41.027229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:41.028435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:41.029481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 633/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:27:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:41.035053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.053301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:41.132753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0756706-53e7-4593-95cc-97fc999a7643 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.169287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.169945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.170226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.170501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.171274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:41.171580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:41.172959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:41.181038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 631/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:41.187663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.254826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:41.291693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd8e9-556d-46c3-a103-10722866efd7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.326939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.327735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.328015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.328288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.339574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:41.341022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:41.358904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:41.360024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:41.361190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:41.361190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:41.361190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:41.361190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:41.361190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 634/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:41.366065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.447237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:41.466337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b1b9d-a354-4fd4-b759-744802d807ff None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.499042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.499700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.499981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.500253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.500971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:41.501276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:41.502665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:41.515331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 632/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:41.521079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.622612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cefc34-56f8-4913-a1f4-e08013dce5d2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.642980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:41.656419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.657216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.657501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.657776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.665501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:41.666986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:41.682477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:41.683630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:41.685531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 635/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:41.691129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.793404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d28d22-5a06-45fa-a510-1d9eb32e4758 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:41.829085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:41.829797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:41.830041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:41.830317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:41.831178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:41.831497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:41.833143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:41.842168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:41.850430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 633/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:41.855955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:41.965052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a850a19-beef-4e97-9619-94d2ff0c82ad None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:42.000660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:42.001444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:42.001721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:42.001990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:42.012230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:42.015568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:42.031258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:42.033490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:42.034700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:42.034700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:42.034700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:42.034700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:42.034700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 636/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:42.038674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:42.137358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c3afb0-09ba-4902-9b37-ba65e5a376e4 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:42.170977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:42.171636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:42.171949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:42.172221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:42.172929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:42.173233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:42.174621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:42.186839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 634/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:42.192476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:42.224158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:42.289862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ce48d-1053-421a-8607-6403563e5d3f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:42.323935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:42.324725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:42.325005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:42.325929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:42.332471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:42.333908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:42.348493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:42.349622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:42.350752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 637/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 14:27:42.357977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:42.417665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:42.459590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49721e-aa35-42bb-b1dd-cd9e51387518 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:42.493799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:42.494399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:42.494732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:42.494997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:42.495779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:42.496090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:42.498208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bf60b-6528-4c1c-b02c-7e2931954734 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:42.510323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 635/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:42.516597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:42.607386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:42.615779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bea128-1359-41a4-bb33-0866fe91e426 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:42.650905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:42.651797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:42.652151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:42.652430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] PATH_INFO: `/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/812d75984e6245e2939c28fbd068b895/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:42.655090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ad4c7896fe34e77910020886bf34d03, user_id=812d75984e6245e2939c28fbd068b895, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:42.656572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd4ad6-f5c8-405c-a7de-d5089389af61 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:42.710654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 638/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Fri Jun 24 14:27:42 2022] PUT /identity/v3/projects/3ad4c7896fe34e77910020886bf34d03/users/812d75984e6245e2939c28fbd068b895/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:27:44.723019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca360bb-157f-4dbf-a8fc-fe1581aed0d7 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:44.723542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca360bb-157f-4dbf-a8fc-fe1581aed0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:44.723542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca360bb-157f-4dbf-a8fc-fe1581aed0d7 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:44.724880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 636/1274] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:44.730015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:44.730392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:44.730591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:44.749198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:44.750372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:44.816058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 639/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:44.839785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca360bb-157f-4dbf-a8fc-fe1581aed0d7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:44.940880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca360bb-157f-4dbf-a8fc-fe1581aed0d7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:44.957662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:44.973723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:44.974443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:44.974749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:44.975021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:44.985347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 14:27:44.992748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:44.993854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:44.994952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 637/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Fri Jun 24 14:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:45.000676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.079721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:45.097794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95606e-906b-4440-bff5-57450b5c5f10 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.131553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:45.138205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:45.143214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 640/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Fri Jun 24 14:27:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:45.148996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.248384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3673096-66fc-4812-aa86-7983af85b8be None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.282632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.283434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.283717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.283989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:45.295391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:45.296841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:45.313385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:45.314423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:45.315566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:45.315566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:45.315566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:45.315566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:45.315566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 638/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:45.320514 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.421920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37926-f8d5-462b-aca5-a59c562a55d6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.455080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.455903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.456191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.456493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:45.457215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:45.457529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:45.458931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:45.471188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 641/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:45.476900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.507024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:45.576500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96b69e-6bc3-424a-8aac-12ee2764c8c6 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.609725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.610539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.610823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.611094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:45.618481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:45.619940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 14:27:45.633823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:45.635163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 14:27:45.636321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:45.638683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 639/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:45.641403 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.645183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:45.740625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40130cc9-60e8-46a7-839a-ae3bb99b013a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.773914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.774506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.774809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.775079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:45.775896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:45.776208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:45.777629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:45.794862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 642/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Fri Jun 24 14:27:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:27:45.800786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:45.891302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:45.900415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1850d6-e742-41b6-8fd9-a1b93afdd752 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:45.936051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:45.936750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:45.937074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:45.937452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.003215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.004664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.093784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:46.103611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 640/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Fri Jun 24 14:27:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 14:27:46.109350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:46.171789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:46.209246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545a0f8-93b6-43e9-b20e-7c022ff30f7f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:46.241139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:46.244811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:46.244811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:46.244811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.256121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.257630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 14:27:46.277531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:46.279214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:46.280161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 14:27:46.280161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 14:27:46.280161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 14:27:46.280161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:46.280389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 643/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:27:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:46.286482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:46.401065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044daa32-110d-4cb2-8b87-04059d8e6f27 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:46.437262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:46.437953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:46.438230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:46.438497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.439281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:46.439567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.440974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.458786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 641/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Fri Jun 24 14:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:46.460095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:46.571192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201e1c-063f-4bae-a69d-40eec8910520 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:46.585606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:46.608222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:46.609028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:46.609313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:46.609589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.616544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.620106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 14:27:46.632205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:46.633132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jun 24 14:27:46.634065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 644/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Fri Jun 24 14:27:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:46.640072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:46.695814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:46.745503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1810041-385e-45ea-aa79-62d36c3eda86 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:46.777580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:46.779020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:46.779325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:46.779620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.780446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:46.780756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.782224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df302ed-00d0-4e1c-83ff-ee3637ef4e92 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.794909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 642/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Fri Jun 24 14:27:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:27:46.801063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:46.898342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f96634b-26c1-4ba6-938e-675d40281837 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:46.932588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:46.933231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:46.933508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:46.933782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:46.934637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] RBAC: Adding query filter params (role.id=4ef72abf5489405bbc82b2318479765b, scope.project.id=3ad4c7896fe34e77910020886bf34d03, user.id=812d75984e6245e2939c28fbd068b895) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:46.934943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:46.936362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9bfec-b8d9-440a-8167-78e7592cf04d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:46.942129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 645/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Fri Jun 24 14:27:46 2022] GET /identity/v3/role_assignments?user.id=812d75984e6245e2939c28fbd068b895&scope.project.id=3ad4c7896fe34e77910020886bf34d03&role.id=4ef72abf5489405bbc82b2318479765b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:49.019355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f381d9-7d0b-4807-9fab-5c70e1822e16 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:49.019889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f381d9-7d0b-4807-9fab-5c70e1822e16 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:49.019889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f381d9-7d0b-4807-9fab-5c70e1822e16 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:49.021253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 643/1288] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:49.026503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:49.026959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:49.027087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:49.044838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:49.046046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:49.078893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:49.115634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 646/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:49.139069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f381d9-7d0b-4807-9fab-5c70e1822e16 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:49.240218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f381d9-7d0b-4807-9fab-5c70e1822e16 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:49.251755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:49.274716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:49.275476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:49.275761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:49.276037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:49.276683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:49.278091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:49.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:49.291728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:49.292789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 644/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:27:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:49.298003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:49.397389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1798a1-3928-461c-8a35-ee40bcc64478 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:49.431231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:49.431958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:49.432244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:49.432517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:49.433217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:49.433570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:49.434983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd6e48-9f1b-4919-8ce4-0d0701bb581e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:49.440579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 647/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Jun 24 14:27:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:49.446500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:49.549420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201b5ae-20ce-4dca-8085-470670e7f68e None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:49.584753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:49.585445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:49.585730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:49.586033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] PATH_INFO: `/v3/services` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:49.586781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:49.587094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:49.588496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1349c-1f1b-47e2-976b-6947fa4d9ba0 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:49.593935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 645/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Jun 24 14:27:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:51.619998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a43cee-a544-41cc-a689-ff0cc911b833 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:51.620509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a43cee-a544-41cc-a689-ff0cc911b833 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:51.620509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a43cee-a544-41cc-a689-ff0cc911b833 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:51.621839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 648/1293] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:51.627829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5134-3761-4810-bfed-f0a2fd7cb7c2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:51.628243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5134-3761-4810-bfed-f0a2fd7cb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:51.628397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e5134-3761-4810-bfed-f0a2fd7cb7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:51.645143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e88e5134-3761-4810-bfed-f0a2fd7cb7c2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:51.646372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e88e5134-3761-4810-bfed-f0a2fd7cb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:51.676122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:51.711777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 646/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:51.735005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a43cee-a544-41cc-a689-ff0cc911b833 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:51.765026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:51.833425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a43cee-a544-41cc-a689-ff0cc911b833 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:51.869212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:51.869869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:51.870198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:51.870425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:51.871242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:51.872621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84795269-e3e6-479e-a2a7-b57b119cedfa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:51.973535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 649/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Fri Jun 24 14:27:51 2022] POST /identity/v3/services => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:27:54.028662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1a66-2055-46f0-a785-04dd202920c5 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.030118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1a66-2055-46f0-a785-04dd202920c5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.030118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e1a66-2055-46f0-a785-04dd202920c5 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.031161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 647/1296] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:27:54.036730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.037142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.037290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.054658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:54.055832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:54.129063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 650/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:54 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:54.152010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e1a66-2055-46f0-a785-04dd202920c5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:54.253576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e1a66-2055-46f0-a785-04dd202920c5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:54.287271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:54.292576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.292576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.292576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.292576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:54.292576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:54.302907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:54.304057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:54.304933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 648/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:27:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:54.310401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:54.328356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:54.410035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c02cc-958f-4b97-82b1-0f5a28831e8b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:54.443662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:54.444326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.444611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.444888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.445594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:54.445930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:54.447326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:54.453393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 651/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Jun 24 14:27:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:54.459314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:54.491318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:54.559781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb45ec-26aa-4237-8921-66e9e7e2767c None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:54.593118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:54.593866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.594147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.594420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.595227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] RBAC: Adding query filter params (interface=public, service_id=478d03fe17cf402d889dcde1426cbba9, region_id=RegionOne) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:54.595536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:54.596889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad855464-4e13-4241-b5fb-99342cbf5143 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:54.601894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 649/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Fri Jun 24 14:27:54 2022] GET /identity/v3/endpoints?service_id=478d03fe17cf402d889dcde1426cbba9&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:54.608142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:54.698011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:54.720076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca6e43-0796-4600-9b04-8aa55c272e27 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:54.754842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:54.755508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:54.755792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:54.756069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:54.756746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:54.758115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d077b-cf84-41e2-b4c6-9b90825f8417 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:54.765948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 652/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Fri Jun 24 14:27:54 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:27:56.791315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e88af-7460-461d-9bb2-14201c4aaa30 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:56.791871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e88af-7460-461d-9bb2-14201c4aaa30 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:56.791871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e88af-7460-461d-9bb2-14201c4aaa30 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:56.795822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 650/1302] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:27:56.799054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:56.799508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:56.799600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:56.816622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:27:56.817787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:27:56.883551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 653/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:27:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 14:27:56.906673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e88af-7460-461d-9bb2-14201c4aaa30 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:57.000493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:57.006939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e88af-7460-461d-9bb2-14201c4aaa30 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:57.039832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:57.040630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:57.041181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:57.041738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:57.042367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:57.044781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 14:27:57.056476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 14:27:57.057661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application with Lock( Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self._enter() Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application return self.creator() Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application created_value = creator( Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: ERROR keystone.server.flask.application Jun 24 14:27:57.058739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 651/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Fri Jun 24 14:27:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 14:27:57.063733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:57.140525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:27:57.163916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531215eb-7479-44dd-b7f6-bec28b2e6b19 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:57.197515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:57.198169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:57.198469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:57.198772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:57.199509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:27:57.199818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:57.201199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa47379-6737-487f-9fe6-066d2fb0334a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:57.206315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 654/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Fri Jun 24 14:27:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:27:57.212102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:27:57.311582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a3b8-a804-42f7-8ce9-32031c2d3797 None admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:27:57.344113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:27:57.344782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:27:57.345065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:27:57.345340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:27:57.346071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:27:57.347460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa1b46-ab6d-4670-8119-1072009d92eb None admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:27:57.417895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 652/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Fri Jun 24 14:27:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:28:07.203261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=30 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:28:07.413142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:34.170167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b7c3-f431-4a90-bea2-0d85ac5b9ad6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:34.170749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b7c3-f431-4a90-bea2-0d85ac5b9ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:34.170749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416b7c3-f431-4a90-bea2-0d85ac5b9ad6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:34.177600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04574bd-37ec-4a2d-820a-159a2bd06a4c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:34.178271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04574bd-37ec-4a2d-820a-159a2bd06a4c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:34.178271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04574bd-37ec-4a2d-820a-159a2bd06a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:34.181517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 655/1307] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:34 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:34.257040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c04574bd-37ec-4a2d-820a-159a2bd06a4c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:34.258272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c04574bd-37ec-4a2d-820a-159a2bd06a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:34.378896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 653/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:34:34.449860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:34.450845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:34.451573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:34.453043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 656/1309] 10.222.0.23 () {54 vars in 964 bytes} [Fri Jun 24 14:34:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:34.467548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a6ad-bdd0-4e29-9ef6-c0b43e24e9f3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:34.467973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a6ad-bdd0-4e29-9ef6-c0b43e24e9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:34.468389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a6ad-bdd0-4e29-9ef6-c0b43e24e9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:34.561714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6c33a6ad-bdd0-4e29-9ef6-c0b43e24e9f3 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:34.562161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c33a6ad-bdd0-4e29-9ef6-c0b43e24e9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:34.586231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:34.659588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 654/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Jun 24 14:34:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:34.669008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:34.671000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:34.814938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51cb70-5010-4143-adae-10ac9fbf7df6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:34.916559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:34.917202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:34.917484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:34.917844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:34.920383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:34.990979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:35.055570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:35.057256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d51a3b-dc09-46a9-aa79-6ed0aa5eb25a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:35.078812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 657/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:34:37.631071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:37.631592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:37.631674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:37.634815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 655/1312] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:37.639042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd498f-9182-4bbe-9cd6-029502e22d21 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:37.639592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd498f-9182-4bbe-9cd6-029502e22d21 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:37.639592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd498f-9182-4bbe-9cd6-029502e22d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:37.727800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6bfd498f-9182-4bbe-9cd6-029502e22d21 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:37.729055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6bfd498f-9182-4bbe-9cd6-029502e22d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:37.840873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 658/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:37.879818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:37.881582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:38.000690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b59923-53b5-4621-ac10-e6f47ffc4dfa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:38.079895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:38.082504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:38.083270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:38.083556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:38.083827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:38.085471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:38.210293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:38.211698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac059a-12a3-48cd-afcf-3ba9689db8c6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:38.234808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 656/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:40.769998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:40.770737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:40.770737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:40.771968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 659/1315] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:40.777049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b86d0-d5f5-4db9-a266-8eaf20db347a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:40.777797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b86d0-d5f5-4db9-a266-8eaf20db347a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:40.777907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b86d0-d5f5-4db9-a266-8eaf20db347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:40.841285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-de7b86d0-d5f5-4db9-a266-8eaf20db347a None None] MFA Rules not processed for user `59184329dff84fd99b155057248ab763`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:40.842567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de7b86d0-d5f5-4db9-a266-8eaf20db347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:40.897643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:40.943922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 657/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:40.984673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:40.986549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:40.998407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:41.103129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089ceff-7db5-4539-b48c-55b2a576cb1d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:41.177694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:41.178339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:41.178640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:41.178910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:41.180553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:41.241781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:41.288987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:41.290429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f05b2-6d17-4114-a1d0-b62641f62f58 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:41.309776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 660/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:44.214101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:44.214707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:44.214707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:44.216670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 658/1318] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:44.221934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec63c-0145-407b-9a2d-edae882da849 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:44.222330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec63c-0145-407b-9a2d-edae882da849 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:44.222523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec63c-0145-407b-9a2d-edae882da849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:44.286488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c67ec63c-0145-407b-9a2d-edae882da849 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:44.287757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c67ec63c-0145-407b-9a2d-edae882da849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:44.383277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 661/1319] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:44.422574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:44.424108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:44.528315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd16d-5290-44e8-b537-58adad211059 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:44.601490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:44.602132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:44.602411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:44.602713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:44.604596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:44.612215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:44.714337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:44.715836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ae040-987b-410d-aeaf-6c6bf976385d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:44.733741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 659/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:47.925562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:47.926134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:47.926239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:47.927651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 662/1321] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:47.933346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0e12-49a6-4a9b-ad0a-2a51ca936f2d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:47.933828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0e12-49a6-4a9b-ad0a-2a51ca936f2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:47.934031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f0e12-49a6-4a9b-ad0a-2a51ca936f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:48.005017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a42f0e12-49a6-4a9b-ad0a-2a51ca936f2d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:48.006255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a42f0e12-49a6-4a9b-ad0a-2a51ca936f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:48.069332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:48.111261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 660/1322] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:48.148558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:48.150130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:48.171516 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:48.265273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75a991-06e7-4ba7-ab7d-d175013964c2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:48.349849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:48.350665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:48.350837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:48.351122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:48.352890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:48.439555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:48.480856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:48.482436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b4f26-ccb1-4058-bbc9-3c1fc1799129 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:48.504657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 663/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:50.644041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:50.644558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:50.644558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:50.645876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 661/1324] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:50.651221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48604aa-fdc8-4f2c-a157-29ef12258d82 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:50.651661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48604aa-fdc8-4f2c-a157-29ef12258d82 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:50.652010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48604aa-fdc8-4f2c-a157-29ef12258d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:50.718200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e48604aa-fdc8-4f2c-a157-29ef12258d82 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:50.719440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e48604aa-fdc8-4f2c-a157-29ef12258d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:50.817844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 664/1325] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:50.852967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:50.854606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:50.969115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3e2da-60d2-4d80-94f1-3131d9a75db4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:51.050801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:51.051454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:51.051760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:51.052035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:51.053713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:51.072814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:51.170713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:51.172182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df8277-5d4c-4485-bb60-58bb02c71342 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:51.193544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 662/1326] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:53.339358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:53.339978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:53.339978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:53.341626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 665/1327] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:53.347388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133effa-8dfa-4fb8-9447-f17a5481364d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:53.348046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133effa-8dfa-4fb8-9447-f17a5481364d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:53.348046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133effa-8dfa-4fb8-9447-f17a5481364d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:53.418482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8133effa-8dfa-4fb8-9447-f17a5481364d None None] MFA Rules not processed for user `59184329dff84fd99b155057248ab763`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:53.420217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8133effa-8dfa-4fb8-9447-f17a5481364d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:53.490195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:53.527103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 663/1328] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:53.561871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:53.563495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:53.596037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:53.681297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59391e-250a-47e2-93c0-196b1c1a57f0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:53.762975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:53.763639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:53.763922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:53.764197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:53.765902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:53.848929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:53.884766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:53.886245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50990c96-9a21-45db-9944-10d9d09bde2b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:53.907971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 666/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:56.539597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:56.540008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:56.540207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:56.541626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 664/1330] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:56.547735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1892-2247-405d-8f03-8bf5a053fbff None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:56.548156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1892-2247-405d-8f03-8bf5a053fbff None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:56.548435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1892-2247-405d-8f03-8bf5a053fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:56.616053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9f5f1892-2247-405d-8f03-8bf5a053fbff None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:56.617329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9f5f1892-2247-405d-8f03-8bf5a053fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:56.722857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 667/1331] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:56.758261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:56.759834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:56.863576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d2476-b5ec-4870-a338-ccccf6b82a9e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:56.937458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:56.938019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:56.938298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:56.938596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:56.940237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:56.968563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:57.051402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:57.052870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d854e76-0e93-4131-9f70-0ef718ba61ad service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:57.071866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 665/1332] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:34:59.219676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:59.220220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:59.220220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:59.221585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 668/1333] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:34:59.227006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133c91-c93e-4df4-8aad-37a3cf22b0bd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:59.227408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133c91-c93e-4df4-8aad-37a3cf22b0bd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:59.227558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e133c91-c93e-4df4-8aad-37a3cf22b0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:59.296799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e133c91-c93e-4df4-8aad-37a3cf22b0bd None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:34:59.298028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e133c91-c93e-4df4-8aad-37a3cf22b0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:59.377614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:59.401748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 666/1334] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:34:59.442481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:34:59.444038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:59.484833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:59.549840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce19e3e-7780-499e-b6a9-d3a11642bc4a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:34:59.625946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:34:59.626629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:34:59.626914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:34:59.627189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:34:59.629052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:34:59.713863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:34:59.740013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:34:59.741391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b5952-9bbc-4f08-8c2a-1456200b1c15 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:34:59.759610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 669/1335] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:02.711957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:02.712467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:02.712467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:02.713810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 667/1336] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:02.719359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26ee8-d803-422c-afc5-2178e246bdb6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:02.719789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26ee8-d803-422c-afc5-2178e246bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:02.719916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b26ee8-d803-422c-afc5-2178e246bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:02.787703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b1b26ee8-d803-422c-afc5-2178e246bdb6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:02.788972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b1b26ee8-d803-422c-afc5-2178e246bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:02.905841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 670/1337] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:02.942759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:02.944697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:03.054439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a3c99-32ca-40a0-978c-53ddd78edb04 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:03.131001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:03.131660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:03.131938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:03.132213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:03.133956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:03.171100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:03.254652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:03.256100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ef342-b705-45a6-8dc0-fd5714240d28 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:03.278247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 668/1338] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:07.826008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d30f-1574-4eff-94a3-03d95e54d28c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:07.826008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d30f-1574-4eff-94a3-03d95e54d28c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:07.826008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05d30f-1574-4eff-94a3-03d95e54d28c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:07.827907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 671/1339] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:07.833575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe3ba7-5a5b-4a2d-95a9-4ed8c4e7bfec None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:07.834071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe3ba7-5a5b-4a2d-95a9-4ed8c4e7bfec None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:07.834180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe3ba7-5a5b-4a2d-95a9-4ed8c4e7bfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:07.921057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7cfe3ba7-5a5b-4a2d-95a9-4ed8c4e7bfec None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:07.926486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7cfe3ba7-5a5b-4a2d-95a9-4ed8c4e7bfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:08.024116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:08.038267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 669/1340] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:08.094397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ef59-1d14-4c0d-8086-650b4f09826d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:08.094788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ef59-1d14-4c0d-8086-650b4f09826d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:08.095011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91ef59-1d14-4c0d-8086-650b4f09826d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:08.096678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 672/1341] 10.222.0.23 () {54 vars in 964 bytes} [Fri Jun 24 14:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:08.107846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97662b-9957-44df-8143-17505c7f156b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:08.108406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97662b-9957-44df-8143-17505c7f156b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:08.108869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97662b-9957-44df-8143-17505c7f156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:08.194187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1c97662b-9957-44df-8143-17505c7f156b None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:08.195338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c97662b-9957-44df-8143-17505c7f156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:08.281406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 670/1342] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Jun 24 14:35:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:08.290409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ef59-1d14-4c0d-8086-650b4f09826d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:08.303219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91ef59-1d14-4c0d-8086-650b4f09826d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:08.306737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:08.307408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:08.307668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:08.307976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:08.317277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:08.318739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da3af8-a851-45cd-b8e8-616ea0abfa70 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:08.328566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 673/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:12.060332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:12.060959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:12.060959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:12.062335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 671/1344] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:12.068490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117827ce-f28d-41e8-afa2-2836cf0e032e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:12.069081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117827ce-f28d-41e8-afa2-2836cf0e032e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:12.069081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117827ce-f28d-41e8-afa2-2836cf0e032e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:12.099051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-117827ce-f28d-41e8-afa2-2836cf0e032e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:12.100299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-117827ce-f28d-41e8-afa2-2836cf0e032e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:12.118696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:12.180358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 674/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:12.219409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:12.221286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:12.234971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bd3af-e142-4387-bb82-9456e7b8310c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:12.238626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:12.239481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:12.239611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:12.239830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:12.249673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:12.251091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec48137-0bcd-4481-b937-3debf7f67878 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:12.261428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 672/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:16.643759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc4d2-26ec-4e28-b85f-b6bc7e27f145 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:16.644215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc4d2-26ec-4e28-b85f-b6bc7e27f145 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:16.644310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bc4d2-26ec-4e28-b85f-b6bc7e27f145 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:16.650793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 675/1347] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:16.652714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390912a-5e20-4c08-85e9-177e2c39b8a2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:16.653151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390912a-5e20-4c08-85e9-177e2c39b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:16.653275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390912a-5e20-4c08-85e9-177e2c39b8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:16.672683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8390912a-5e20-4c08-85e9-177e2c39b8a2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:16.674023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8390912a-5e20-4c08-85e9-177e2c39b8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:16.689299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 673/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:35:16.727673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc4d2-26ec-4e28-b85f-b6bc7e27f145 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:16.741243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bc4d2-26ec-4e28-b85f-b6bc7e27f145 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:16.745267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:16.746006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:16.746295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:16.746601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:16.756697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:16.758275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b7e65-aa52-448f-886e-d8da9981aff8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:16.769229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 676/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:21.877345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326f240-9300-4dfd-a067-a7480dfdd5b6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:21.877850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326f240-9300-4dfd-a067-a7480dfdd5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:21.877946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326f240-9300-4dfd-a067-a7480dfdd5b6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:21.879580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 674/1350] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:21.902232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed4853-340c-438c-b238-a211b0c3e16c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:21.902232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed4853-340c-438c-b238-a211b0c3e16c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:21.902232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed4853-340c-438c-b238-a211b0c3e16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:21.941293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fbed4853-340c-438c-b238-a211b0c3e16c None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:21.942912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fbed4853-340c-438c-b238-a211b0c3e16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:22.080339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 677/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:35:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:26.573207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de8ee-6a31-4a5c-ab65-5e3c76559033 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:26.574341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de8ee-6a31-4a5c-ab65-5e3c76559033 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:26.574341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758de8ee-6a31-4a5c-ab65-5e3c76559033 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:26.575685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 675/1352] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:26.580859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a064-ea53-4c80-8872-61c21a428192 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:26.581292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a064-ea53-4c80-8872-61c21a428192 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:26.581423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724a064-ea53-4c80-8872-61c21a428192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:26.600749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3724a064-ea53-4c80-8872-61c21a428192 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:26.607419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3724a064-ea53-4c80-8872-61c21a428192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:26.612377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 678/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:35:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:26.667193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=66 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:28.179078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968607-2692-495c-829e-de0b2d5e5f6d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:28.179733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968607-2692-495c-829e-de0b2d5e5f6d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:28.179733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2968607-2692-495c-829e-de0b2d5e5f6d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:28.181198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 676/1354] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:35:28.187338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f42c-9768-4d19-bb61-a4908fd4fe39 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:28.187865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f42c-9768-4d19-bb61-a4908fd4fe39 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:28.187958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15f42c-9768-4d19-bb61-a4908fd4fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:28.206753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1d15f42c-9768-4d19-bb61-a4908fd4fe39 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:28.207986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1d15f42c-9768-4d19-bb61-a4908fd4fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:28.217179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 679/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:32.234836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcdec-1bc4-463d-88c3-6e5045269acd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:32.235786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcdec-1bc4-463d-88c3-6e5045269acd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:32.236292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bcdec-1bc4-463d-88c3-6e5045269acd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:32.243193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 677/1356] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:32.245191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000241ff-f5ca-4b4c-95e6-c5a69e34d14d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:32.249907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000241ff-f5ca-4b4c-95e6-c5a69e34d14d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:32.250470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000241ff-f5ca-4b4c-95e6-c5a69e34d14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:32.285551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-000241ff-f5ca-4b4c-95e6-c5a69e34d14d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:32.287477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-000241ff-f5ca-4b4c-95e6-c5a69e34d14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:32.297840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 680/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:32.368454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df05d6-e6aa-4a6a-a550-2af4cec27a32 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:32.368720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df05d6-e6aa-4a6a-a550-2af4cec27a32 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:32.370454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df05d6-e6aa-4a6a-a550-2af4cec27a32 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:32.373910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 678/1358] 10.222.0.23 () {54 vars in 961 bytes} [Fri Jun 24 14:35:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:32.381895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea289-d02f-4f95-a085-2c592c209fd0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:32.382804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea289-d02f-4f95-a085-2c592c209fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:32.383093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ea289-d02f-4f95-a085-2c592c209fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:32.437602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c73ea289-d02f-4f95-a085-2c592c209fd0 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:32.438872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c73ea289-d02f-4f95-a085-2c592c209fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:32.446498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:32.533421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 681/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Jun 24 14:35:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:35:32.543111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df05d6-e6aa-4a6a-a550-2af4cec27a32 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:32.559280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df05d6-e6aa-4a6a-a550-2af4cec27a32 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:32.565653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:32.570133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:32.570133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:32.570133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:32.577944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:32.579626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7efaaf-0299-4ad3-8069-718f38cf01a8 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:32.590762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 679/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:34.978951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c64f-46cf-4fcf-99dd-e1bb0ef37c6f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:34.979490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c64f-46cf-4fcf-99dd-e1bb0ef37c6f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:34.979490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c64f-46cf-4fcf-99dd-e1bb0ef37c6f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:34.980813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 682/1361] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:34.986664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e234b7-797c-48bb-a75d-73f446fc33f0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:34.987198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e234b7-797c-48bb-a75d-73f446fc33f0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:34.987457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e234b7-797c-48bb-a75d-73f446fc33f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:35.007106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d0e234b7-797c-48bb-a75d-73f446fc33f0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:35.008483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d0e234b7-797c-48bb-a75d-73f446fc33f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:35.017901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 680/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:35.069436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c64f-46cf-4fcf-99dd-e1bb0ef37c6f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:35.086843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84c64f-46cf-4fcf-99dd-e1bb0ef37c6f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:35.097359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:35.099919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:35.100279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:35.100279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:35.110831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:35.112480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30627e6c-17b4-42ef-ae30-c7ed409ceadd service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:35.123608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 683/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:37.748562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c7a5-ebd6-4d0e-9431-684e4a547a2c None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:37.749135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c7a5-ebd6-4d0e-9431-684e4a547a2c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:37.749359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c7a5-ebd6-4d0e-9431-684e4a547a2c None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:37.751441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 681/1364] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:35:37.757613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc8d8-6954-42be-9847-56ba873e6ddb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:37.758044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc8d8-6954-42be-9847-56ba873e6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:37.758373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abc8d8-6954-42be-9847-56ba873e6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:37.779682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-05abc8d8-6954-42be-9847-56ba873e6ddb None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:37.781138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-05abc8d8-6954-42be-9847-56ba873e6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:37.794909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 684/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:37.846880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5aaf-f511-4d8d-be2c-16ff08758594 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:37.847197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5aaf-f511-4d8d-be2c-16ff08758594 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:37.847450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5aaf-f511-4d8d-be2c-16ff08758594 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:37.849541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 682/1366] 10.222.0.23 () {54 vars in 961 bytes} [Fri Jun 24 14:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:37.856335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633e85-c11a-4573-b7cb-7fc3b7c5a789 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:37.856617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633e85-c11a-4573-b7cb-7fc3b7c5a789 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:37.857097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d633e85-c11a-4573-b7cb-7fc3b7c5a789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:37.877487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7d633e85-c11a-4573-b7cb-7fc3b7c5a789 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:37.878842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7d633e85-c11a-4573-b7cb-7fc3b7c5a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:37.888229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 685/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Jun 24 14:35:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:37.896727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5aaf-f511-4d8d-be2c-16ff08758594 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:37.912607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5aaf-f511-4d8d-be2c-16ff08758594 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:37.919521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:37.919914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:37.920201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:37.920476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:37.935679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:37.937473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092f436-c991-4bd5-928f-baf5c66f6dc7 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:37.948407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 683/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:40.362503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7130433-f7f1-4692-9660-b48ed6ef5078 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:40.363580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7130433-f7f1-4692-9660-b48ed6ef5078 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:40.364069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7130433-f7f1-4692-9660-b48ed6ef5078 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:40.365676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 686/1369] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:40.372341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08f21e-f4dc-4647-bc09-8394b2848d4f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:40.373096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08f21e-f4dc-4647-bc09-8394b2848d4f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:40.373096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08f21e-f4dc-4647-bc09-8394b2848d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:40.392846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e08f21e-f4dc-4647-bc09-8394b2848d4f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:40.394126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e08f21e-f4dc-4647-bc09-8394b2848d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:40.405975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 684/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:40.458728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7130433-f7f1-4692-9660-b48ed6ef5078 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:40.475549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7130433-f7f1-4692-9660-b48ed6ef5078 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:40.481746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:40.482547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:40.482861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:40.483138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:40.492539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:40.494153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5afbd5-8465-4c3a-ac4f-20f9d22d9bab service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:40.505829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 687/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:41.134213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807e41-e969-469f-a158-72e00994610b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:41.135303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807e41-e969-469f-a158-72e00994610b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:41.135303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807e41-e969-469f-a158-72e00994610b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:41.136374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 685/1372] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:41.144527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0a47-3632-42f4-89f6-23b7a77be614 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:41.148814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0a47-3632-42f4-89f6-23b7a77be614 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:41.148814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a0a47-3632-42f4-89f6-23b7a77be614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:41.166482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f28a0a47-3632-42f4-89f6-23b7a77be614 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:41.168026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f28a0a47-3632-42f4-89f6-23b7a77be614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:41.179067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 688/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:41.540907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f554a4-e68e-4626-a9eb-074d6be0d1f6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:41.540907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f554a4-e68e-4626-a9eb-074d6be0d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:41.540907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f554a4-e68e-4626-a9eb-074d6be0d1f6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:41.540907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 686/1374] 10.222.0.23 () {54 vars in 963 bytes} [Fri Jun 24 14:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:41.545765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5bd9-0533-4684-8357-78cead0c022e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:41.546271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5bd9-0533-4684-8357-78cead0c022e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:41.546426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b5bd9-0533-4684-8357-78cead0c022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:41.602887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f99b5bd9-0533-4684-8357-78cead0c022e None None] MFA Rules not processed for user `c42ba8788bf2431a96b2087b287d7426`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:41.604249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f99b5bd9-0533-4684-8357-78cead0c022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:41.673090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 689/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Fri Jun 24 14:35:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:41.680429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f554a4-e68e-4626-a9eb-074d6be0d1f6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:41.692576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f554a4-e68e-4626-a9eb-074d6be0d1f6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:41.697142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:41.698342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:41.698815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:41.699275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:41.709973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:41.711457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e03b-4556-46a3-8704-8cae9e265109 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:41.722555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 687/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:42.848827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc510a-a514-4d73-b78c-1550bd6aa793 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:42.849417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc510a-a514-4d73-b78c-1550bd6aa793 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:42.849417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc510a-a514-4d73-b78c-1550bd6aa793 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:42.850777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 690/1377] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:42.856245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fce08-67bb-4bc6-a454-5317bc635e71 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:42.856713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fce08-67bb-4bc6-a454-5317bc635e71 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:42.856813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fce08-67bb-4bc6-a454-5317bc635e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:42.875304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c91fce08-67bb-4bc6-a454-5317bc635e71 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:42.876524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c91fce08-67bb-4bc6-a454-5317bc635e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:42.885014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 688/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:42.932105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc510a-a514-4d73-b78c-1550bd6aa793 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:42.947942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc510a-a514-4d73-b78c-1550bd6aa793 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:42.953335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:42.953981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:42.954259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:42.954606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:42.963765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:42.965386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cd428-8a3f-40e7-ac04-026a6b52afd1 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:42.976190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 691/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:45.219094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3cdd-7b95-449e-93f2-4f73a02b6b12 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:45.219808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3cdd-7b95-449e-93f2-4f73a02b6b12 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:45.219808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a3cdd-7b95-449e-93f2-4f73a02b6b12 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:45.221274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 689/1380] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:45.228315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dd6b7-097e-4907-8f4e-9d3956faa679 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:45.228838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dd6b7-097e-4907-8f4e-9d3956faa679 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:45.229162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168dd6b7-097e-4907-8f4e-9d3956faa679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:45.239768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:45.250813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-168dd6b7-097e-4907-8f4e-9d3956faa679 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:45.252108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-168dd6b7-097e-4907-8f4e-9d3956faa679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:45.261164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 692/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:45.306718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a3cdd-7b95-449e-93f2-4f73a02b6b12 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:45.327235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a3cdd-7b95-449e-93f2-4f73a02b6b12 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:45.333279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:45.333925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:45.334206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:45.334479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:45.348006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:45.350431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f09335-0deb-44da-aca8-30582f3857bc service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:45.362624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 690/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:47.742965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9caba-20b7-40aa-af2a-2b02ce579724 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:47.744839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9caba-20b7-40aa-af2a-2b02ce579724 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:47.745292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9caba-20b7-40aa-af2a-2b02ce579724 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:47.751384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 693/1383] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:35:47.752869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20d73-1a23-4dba-9b8c-032fe874d665 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:47.753343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20d73-1a23-4dba-9b8c-032fe874d665 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:47.753437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c20d73-1a23-4dba-9b8c-032fe874d665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:47.773761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-85c20d73-1a23-4dba-9b8c-032fe874d665 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:47.775566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-85c20d73-1a23-4dba-9b8c-032fe874d665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:47.784761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 691/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:47.854367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9caba-20b7-40aa-af2a-2b02ce579724 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:47.876096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9caba-20b7-40aa-af2a-2b02ce579724 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:47.882421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:47.883209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:47.883614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:47.884816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:47.895187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:47.896895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094eba8-5008-4fa6-9342-0e39698f74e9 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:47.910123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 694/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:35:50.510002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3be1-c2ce-45ae-ae5c-741d8a31a317 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:50.510583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3be1-c2ce-45ae-ae5c-741d8a31a317 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:50.510750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e3be1-c2ce-45ae-ae5c-741d8a31a317 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:50.512657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 692/1386] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:50.518938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3fb3c-c476-480b-8195-0e1ef48a3973 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:50.518938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3fb3c-c476-480b-8195-0e1ef48a3973 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:50.518938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3fb3c-c476-480b-8195-0e1ef48a3973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:50.541037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9fd3fb3c-c476-480b-8195-0e1ef48a3973 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:50.542260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9fd3fb3c-c476-480b-8195-0e1ef48a3973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:50.550772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 695/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:50.600540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e3be1-c2ce-45ae-ae5c-741d8a31a317 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:50.623075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e3be1-c2ce-45ae-ae5c-741d8a31a317 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:50.629708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:50.630355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:50.630671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:50.630948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:50.640486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:50.641980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81cd1e-bde1-49ae-8fb1-9b84bbab83fd service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:50.655762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 693/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:53.025588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c7a3-e944-4d5e-a049-cf0596b88b4e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:53.026256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c7a3-e944-4d5e-a049-cf0596b88b4e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:53.026256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22c7a3-e944-4d5e-a049-cf0596b88b4e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:53.027735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 696/1389] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:53.033463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d247-bf34-47dc-9740-4ad5acb1ae8b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:53.034097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d247-bf34-47dc-9740-4ad5acb1ae8b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:53.034097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d247-bf34-47dc-9740-4ad5acb1ae8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:53.052833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2587d247-bf34-47dc-9740-4ad5acb1ae8b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:53.054087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2587d247-bf34-47dc-9740-4ad5acb1ae8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:53.064030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 694/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:53.111220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22c7a3-e944-4d5e-a049-cf0596b88b4e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:53.127414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22c7a3-e944-4d5e-a049-cf0596b88b4e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:53.134086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:53.135142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:53.135462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:53.135702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:53.145309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:53.146986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac2395-59be-40e7-a4cb-e23a275ade8a service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:53.159298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 697/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:35:56.005981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7726-e6e5-4e7b-82ce-b8bff52e1478 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:56.006570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7726-e6e5-4e7b-82ce-b8bff52e1478 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:56.006570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb7726-e6e5-4e7b-82ce-b8bff52e1478 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:56.007931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 695/1392] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:56.013955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbb33-d7b0-4ac9-9043-838b09b03bef None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:56.014446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbb33-d7b0-4ac9-9043-838b09b03bef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:56.014571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dbbb33-d7b0-4ac9-9043-838b09b03bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:56.033570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-81dbbb33-d7b0-4ac9-9043-838b09b03bef None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:56.034810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-81dbbb33-d7b0-4ac9-9043-838b09b03bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:56.051657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 698/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:56.099808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7726-e6e5-4e7b-82ce-b8bff52e1478 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:56.118644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb7726-e6e5-4e7b-82ce-b8bff52e1478 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:56.126789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:56.127730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:56.128147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:56.128575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:56.142396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:56.145000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901df9f-0739-4a3b-ae5c-829b20a85e37 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:56.160078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 696/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:58.048134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab9d7-f913-4033-a8d5-aa27d5493bbe None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.048940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab9d7-f913-4033-a8d5-aa27d5493bbe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.048940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ab9d7-f913-4033-a8d5-aa27d5493bbe None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.050234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 699/1395] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:35:58.056422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c2a4-864a-4eda-822e-583069c77008 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.056940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c2a4-864a-4eda-822e-583069c77008 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.057079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c2a4-864a-4eda-822e-583069c77008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.063419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=52 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:58.077318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8bc5c2a4-864a-4eda-822e-583069c77008 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:58.078569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8bc5c2a4-864a-4eda-822e-583069c77008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:58.088510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 697/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:58.150669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0bc35-6493-49ba-89dc-c0a8d50c1fc4 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.150969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0bc35-6493-49ba-89dc-c0a8d50c1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.151246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0bc35-6493-49ba-89dc-c0a8d50c1fc4 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.152651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 700/1397] 10.222.0.23 () {54 vars in 962 bytes} [Fri Jun 24 14:35:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:58.159748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527f2b-ccef-4f1a-9504-f1337f812ba0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.160178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527f2b-ccef-4f1a-9504-f1337f812ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.160696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2527f2b-ccef-4f1a-9504-f1337f812ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.226274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c2527f2b-ccef-4f1a-9504-f1337f812ba0 None None] MFA Rules not processed for user `bb628c1d9c854cbca433d36d143374c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:58.227614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2527f2b-ccef-4f1a-9504-f1337f812ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:58.307683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 698/1398] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Jun 24 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:58.314325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0bc35-6493-49ba-89dc-c0a8d50c1fc4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:58.327020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b0bc35-6493-49ba-89dc-c0a8d50c1fc4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:58.327020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:58.327020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.327020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.334458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.342456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:58.344047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf18b04-144c-4f3b-a135-bdd3b1dfba96 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:58.356157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 701/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:58.569087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd17c-09ad-4812-b76c-058315b9e3d2 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.569448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd17c-09ad-4812-b76c-058315b9e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.569714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bd17c-09ad-4812-b76c-058315b9e3d2 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.571273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 699/1400] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:35:58.578076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba46-638a-4764-b099-ef9ae35dda09 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.578625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba46-638a-4764-b099-ef9ae35dda09 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.578953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924ba46-638a-4764-b099-ef9ae35dda09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.603836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2924ba46-638a-4764-b099-ef9ae35dda09 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:58.605092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2924ba46-638a-4764-b099-ef9ae35dda09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:58.623000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 702/1401] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:58.675114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bd17c-09ad-4812-b76c-058315b9e3d2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:58.693425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bd17c-09ad-4812-b76c-058315b9e3d2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:58.699783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:58.700422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:58.700706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:58.701019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:58.710030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:58.718101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763114-7068-4d24-9069-4640e41d8148 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:58.721768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 700/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:35:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:35:59.294879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a89-403d-4948-b559-27fc1c19f951 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:59.295493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a89-403d-4948-b559-27fc1c19f951 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:59.295493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7206a89-403d-4948-b559-27fc1c19f951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:59.353805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d7206a89-403d-4948-b559-27fc1c19f951 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:59.356128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d7206a89-403d-4948-b559-27fc1c19f951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:59.436478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 703/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:59.485266 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d8a01-f277-40e2-a498-02689865e6fd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:59.485786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d8a01-f277-40e2-a498-02689865e6fd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:59.485901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d8a01-f277-40e2-a498-02689865e6fd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:59.487256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 701/1404] 10.222.0.23 () {54 vars in 942 bytes} [Fri Jun 24 14:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:35:59.494547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932713e4-54fa-41c0-b30f-8fd3f0760648 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:59.494833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932713e4-54fa-41c0-b30f-8fd3f0760648 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:59.495111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932713e4-54fa-41c0-b30f-8fd3f0760648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:59.545376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-932713e4-54fa-41c0-b30f-8fd3f0760648 None None] MFA Rules not processed for user `52486f194961487ba574eebe49869a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:35:59.546607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-932713e4-54fa-41c0-b30f-8fd3f0760648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:35:59.589012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:35:59.619384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 704/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Fri Jun 24 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:35:59.629026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d8a01-f277-40e2-a498-02689865e6fd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:35:59.643529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d8a01-f277-40e2-a498-02689865e6fd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:35:59.649394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:35:59.650060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:35:59.650368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:35:59.650676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:35:59.659686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:35:59.661274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c10201-0f5e-4f09-88c5-d45b36014975 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:35:59.670273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 702/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:36:01.050248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec6bf-f0da-46a6-96fc-d98fc54a3895 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:01.050857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec6bf-f0da-46a6-96fc-d98fc54a3895 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:01.050857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ec6bf-f0da-46a6-96fc-d98fc54a3895 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:01.052204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 705/1407] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:01.057603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06907-caf0-4ea1-a6e6-add4e9f3b862 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:01.058052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06907-caf0-4ea1-a6e6-add4e9f3b862 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:01.058174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f06907-caf0-4ea1-a6e6-add4e9f3b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:01.078696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d4f06907-caf0-4ea1-a6e6-add4e9f3b862 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:01.079957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d4f06907-caf0-4ea1-a6e6-add4e9f3b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:01.088582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 703/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:01.140089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec6bf-f0da-46a6-96fc-d98fc54a3895 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:01.156329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ec6bf-f0da-46a6-96fc-d98fc54a3895 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:01.162651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:01.163306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:01.163588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:01.163870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:01.177131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:01.179184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc67e9-f3d9-4d1d-a2ff-f23a2a134078 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:01.190851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 706/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:11.071953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=55 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:36:11.189160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=18 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:36:24.489540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ff61-d9cd-458a-ab69-d304ca5d65c6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:24.490104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ff61-d9cd-458a-ab69-d304ca5d65c6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:24.490104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9ff61-d9cd-458a-ab69-d304ca5d65c6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:24.492296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 704/1410] 10.222.0.23 () {54 vars in 918 bytes} [Fri Jun 24 14:36:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:24.497628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756ee3-139c-48e3-bbd1-ea0400a911fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:24.498040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756ee3-139c-48e3-bbd1-ea0400a911fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:24.498173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9756ee3-139c-48e3-bbd1-ea0400a911fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:24.517823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a9756ee3-139c-48e3-bbd1-ea0400a911fa None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:24.519063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a9756ee3-139c-48e3-bbd1-ea0400a911fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:24.527977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 707/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Jun 24 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:24.545758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bfbb1-3fbf-4a9d-8b5f-b44118586d81 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:24.546040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bfbb1-3fbf-4a9d-8b5f-b44118586d81 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:24.546313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bfbb1-3fbf-4a9d-8b5f-b44118586d81 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:24.547720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 705/1412] 10.222.0.23 () {54 vars in 918 bytes} [Fri Jun 24 14:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:24.552567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633260e5-5ea9-4406-ab27-84e857831747 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:24.552853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633260e5-5ea9-4406-ab27-84e857831747 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:24.553124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633260e5-5ea9-4406-ab27-84e857831747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:24.573954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-633260e5-5ea9-4406-ab27-84e857831747 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:24.575206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-633260e5-5ea9-4406-ab27-84e857831747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:24.591894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 708/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Fri Jun 24 14:36:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:27.149734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03dd8-dee7-427f-912e-ee17d470bcb9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:27.150328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03dd8-dee7-427f-912e-ee17d470bcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:27.150328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f03dd8-dee7-427f-912e-ee17d470bcb9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:27.151679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:27.157747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0a3d-789d-487f-b88b-3268914b0805 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:27.158491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0a3d-789d-487f-b88b-3268914b0805 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:27.158491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0a3d-789d-487f-b88b-3268914b0805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:27.177713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-545a0a3d-789d-487f-b88b-3268914b0805 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:27.179606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-545a0a3d-789d-487f-b88b-3268914b0805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:27.187593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:36:27.231778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f03dd8-dee7-427f-912e-ee17d470bcb9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:27.247576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f03dd8-dee7-427f-912e-ee17d470bcb9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:27.253340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:27.253981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:27.254264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:27.254569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:27.263642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:27.265235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae3273-99cd-47a3-becc-b39e795487a9 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:27.275327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 707/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:28.201168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7643a-bdc9-479f-9b79-47b284175bec None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:28.201808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7643a-bdc9-479f-9b79-47b284175bec None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:28.201808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7643a-bdc9-479f-9b79-47b284175bec None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:28.203130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:28.208284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3819-4c3c-41dd-8465-923077990742 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:28.208713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3819-4c3c-41dd-8465-923077990742 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:28.208815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3819-4c3c-41dd-8465-923077990742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:28.227853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7ddf3819-4c3c-41dd-8465-923077990742 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:28.229050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7ddf3819-4c3c-41dd-8465-923077990742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:28.237013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:28.391529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7643a-bdc9-479f-9b79-47b284175bec None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:28.406963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7643a-bdc9-479f-9b79-47b284175bec None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:28.412223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:28.412971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:28.413251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:28.413521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:28.421683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:28.423307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a0516-22db-4cc4-889e-10567cee2dcd service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:28.433884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 711/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:29.329855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93c45-0304-40a8-a4fc-c186e09ff4f9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:29.330435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93c45-0304-40a8-a4fc-c186e09ff4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:29.330435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93c45-0304-40a8-a4fc-c186e09ff4f9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:29.331796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:29.337577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50921-1434-4881-8c5d-1222328c9ff3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:29.338000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50921-1434-4881-8c5d-1222328c9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:29.338130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a50921-1434-4881-8c5d-1222328c9ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:29.355924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-14a50921-1434-4881-8c5d-1222328c9ff3 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:29.357160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14a50921-1434-4881-8c5d-1222328c9ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:29.365090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:29.518590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d93c45-0304-40a8-a4fc-c186e09ff4f9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:29.535437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d93c45-0304-40a8-a4fc-c186e09ff4f9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:29.542164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:29.542854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:29.543140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:29.543416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:29.552901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:29.554977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7bbe6-0945-4ec0-8677-b1319bba1021 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:29.566478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 710/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:30.471192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8aaf8-adb6-4da2-afd3-181115d87c93 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:30.471791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8aaf8-adb6-4da2-afd3-181115d87c93 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:30.471791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8aaf8-adb6-4da2-afd3-181115d87c93 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:30.473200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:30.478389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca5dac-521c-4855-9f75-56e16b10c398 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:30.478894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca5dac-521c-4855-9f75-56e16b10c398 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:30.479006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca5dac-521c-4855-9f75-56e16b10c398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:30.497321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08ca5dac-521c-4855-9f75-56e16b10c398 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:30.498555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08ca5dac-521c-4855-9f75-56e16b10c398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:30.506767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:30.660888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8aaf8-adb6-4da2-afd3-181115d87c93 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:30.676058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8aaf8-adb6-4da2-afd3-181115d87c93 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:30.683243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:30.683927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:30.684651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:30.684651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:30.694319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:30.696075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3dbfe-579d-4107-8dee-c58f36d56e67 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:30.705427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 714/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:31.578314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd81a-4d3e-406a-a6bd-9f8011bfb2c4 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:31.578906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd81a-4d3e-406a-a6bd-9f8011bfb2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:31.578906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294fd81a-4d3e-406a-a6bd-9f8011bfb2c4 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:31.580263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:31.585798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02ea5-5d41-4009-8890-6b0a622cd981 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:31.586273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02ea5-5d41-4009-8890-6b0a622cd981 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:31.586389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c02ea5-5d41-4009-8890-6b0a622cd981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:31.604388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-33c02ea5-5d41-4009-8890-6b0a622cd981 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:31.605595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-33c02ea5-5d41-4009-8890-6b0a622cd981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:31.613719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:31.768023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fd81a-4d3e-406a-a6bd-9f8011bfb2c4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:31.784327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294fd81a-4d3e-406a-a6bd-9f8011bfb2c4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:31.790704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:31.791350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:31.791673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:31.791954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:31.800460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:31.802082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33297273-35fa-4571-8a09-988d21f581d6 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:31.812893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 713/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:32.704167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3906fd-6656-49ae-9f0f-58cbdc8a1c80 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:32.704764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3906fd-6656-49ae-9f0f-58cbdc8a1c80 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:32.704764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3906fd-6656-49ae-9f0f-58cbdc8a1c80 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:32.706149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:32.711273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1dc32-c684-4630-b7cd-aedecdde3916 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:32.711734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1dc32-c684-4630-b7cd-aedecdde3916 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:32.711848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1dc32-c684-4630-b7cd-aedecdde3916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:32.730036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7dc1dc32-c684-4630-b7cd-aedecdde3916 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:32.731282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7dc1dc32-c684-4630-b7cd-aedecdde3916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:32.739357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:32.891004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3906fd-6656-49ae-9f0f-58cbdc8a1c80 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:32.905902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3906fd-6656-49ae-9f0f-58cbdc8a1c80 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:32.911634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:32.920655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:32.920655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:32.920655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:32.922384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:32.924015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431cf11-be91-4b9e-a045-ed6fedda5999 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:32.934461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 717/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:33.864101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ae90-546c-4b92-b7a0-fb55665076e6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:33.864687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ae90-546c-4b92-b7a0-fb55665076e6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:33.864687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ae90-546c-4b92-b7a0-fb55665076e6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:33.866632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:33.872260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9e956-edb2-489a-ad21-298ac48ff44c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:33.872698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9e956-edb2-489a-ad21-298ac48ff44c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:33.872809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9e956-edb2-489a-ad21-298ac48ff44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:33.891635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-86f9e956-edb2-489a-ad21-298ac48ff44c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:33.892866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-86f9e956-edb2-489a-ad21-298ac48ff44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:33.902016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:34.056416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ae90-546c-4b92-b7a0-fb55665076e6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:34.071417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ae90-546c-4b92-b7a0-fb55665076e6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:34.076831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:34.077466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:34.077745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:34.078014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:34.086499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:34.088117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093acfab-7195-4162-9749-5ef076b35a47 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:34.097724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 716/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:35.036439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125f59-dfdc-44b4-96cb-37fc1501e4e0 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:35.036996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125f59-dfdc-44b4-96cb-37fc1501e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:35.037124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4125f59-dfdc-44b4-96cb-37fc1501e4e0 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:35.038534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:35.043585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8ca9-79c6-41fa-a2c5-1bc79eea3b52 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:35.044028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8ca9-79c6-41fa-a2c5-1bc79eea3b52 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:35.044124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b8ca9-79c6-41fa-a2c5-1bc79eea3b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:35.062457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a15b8ca9-79c6-41fa-a2c5-1bc79eea3b52 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:35.063697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a15b8ca9-79c6-41fa-a2c5-1bc79eea3b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:35.073331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:35.225155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4125f59-dfdc-44b4-96cb-37fc1501e4e0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:35.239441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4125f59-dfdc-44b4-96cb-37fc1501e4e0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:35.245148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:35.245809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:35.246090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:35.246407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:35.254655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:35.257037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fd632-4503-46b4-8785-c14d90a5dabf service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:35.266702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 720/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:36.149052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014705-fdda-431a-8774-2276fbf2cece None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:36.149629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014705-fdda-431a-8774-2276fbf2cece None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:36.149629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81014705-fdda-431a-8774-2276fbf2cece None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:36.150995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:36.156010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d613-53da-402b-ab66-c94939aaa4cf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:36.156473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d613-53da-402b-ab66-c94939aaa4cf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:36.156571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999d613-53da-402b-ab66-c94939aaa4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:36.174303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f999d613-53da-402b-ab66-c94939aaa4cf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:36.175515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f999d613-53da-402b-ab66-c94939aaa4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:36.183760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:36.334988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81014705-fdda-431a-8774-2276fbf2cece None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:36.349480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81014705-fdda-431a-8774-2276fbf2cece None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:36.354842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:36.355483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:36.355767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:36.356042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:36.364747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:36.367128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19dfdb-d0e7-4d3d-b816-b634d9821a0f service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:36.376846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 719/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:38.351902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6752fe-754e-46ca-ba59-fe62d60e9f0b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:38.352897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6752fe-754e-46ca-ba59-fe62d60e9f0b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:38.353184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6752fe-754e-46ca-ba59-fe62d60e9f0b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:38.354667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:38.360388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ce5a-bb8b-47d0-835e-df40ddb8c44d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:38.360849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ce5a-bb8b-47d0-835e-df40ddb8c44d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:38.360957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840ce5a-bb8b-47d0-835e-df40ddb8c44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:38.379434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e840ce5a-bb8b-47d0-835e-df40ddb8c44d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:38.380711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e840ce5a-bb8b-47d0-835e-df40ddb8c44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:38.389012 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:38.429182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6752fe-754e-46ca-ba59-fe62d60e9f0b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:38.440405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6752fe-754e-46ca-ba59-fe62d60e9f0b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:38.444373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:38.445040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:38.445300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:38.445575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:38.455056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:38.456493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0826a-043c-424b-82bc-4797f0d8a7ff service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:38.466583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 723/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:40.678315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81c86-31f0-453e-a551-15dac0f7fd1d None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:40.678923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81c86-31f0-453e-a551-15dac0f7fd1d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:40.678923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e81c86-31f0-453e-a551-15dac0f7fd1d None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:40.680223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:40.685647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6189-aad5-45a6-8275-a8599d80d9a1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:40.686116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6189-aad5-45a6-8275-a8599d80d9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:40.686222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca6189-aad5-45a6-8275-a8599d80d9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:40.704216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b2ca6189-aad5-45a6-8275-a8599d80d9a1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:40.705398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b2ca6189-aad5-45a6-8275-a8599d80d9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:40.714151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:36:40.752545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e81c86-31f0-453e-a551-15dac0f7fd1d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:40.763299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e81c86-31f0-453e-a551-15dac0f7fd1d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:40.766792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:40.767428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:40.767711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:40.767991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:40.777505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:40.779009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b66b3d-5dee-4d40-8630-de8f2826f1a9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:40.789375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 722/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:43.276743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d23-697b-46d2-94f4-94b624d54361 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:43.277307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d23-697b-46d2-94f4-94b624d54361 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:43.277307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3551d23-697b-46d2-94f4-94b624d54361 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:43.278698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:36:43.283844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b946ae-9187-4261-abd7-a245c7dd7374 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:43.284285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b946ae-9187-4261-abd7-a245c7dd7374 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:43.284409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b946ae-9187-4261-abd7-a245c7dd7374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:43.302351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-67b946ae-9187-4261-abd7-a245c7dd7374 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:43.303587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-67b946ae-9187-4261-abd7-a245c7dd7374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:43.311647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:43.354426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551d23-697b-46d2-94f4-94b624d54361 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:43.366031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3551d23-697b-46d2-94f4-94b624d54361 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:43.369657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:43.370299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:43.370611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:43.370885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:43.381080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:43.382554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e44d42-a128-447a-8d91-fcb8a26304a2 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:43.392051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 726/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:45.586066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292a83-6a3e-4d30-b813-587f5e341757 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:45.586647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292a83-6a3e-4d30-b813-587f5e341757 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:45.586647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292a83-6a3e-4d30-b813-587f5e341757 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:45.588111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:45.593421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402a22-86f6-4c75-9f59-e08606573358 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:45.593842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402a22-86f6-4c75-9f59-e08606573358 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:45.593947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be402a22-86f6-4c75-9f59-e08606573358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:45.611783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-be402a22-86f6-4c75-9f59-e08606573358 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:45.613027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-be402a22-86f6-4c75-9f59-e08606573358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:45.621619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:45.660168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c292a83-6a3e-4d30-b813-587f5e341757 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:45.671469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c292a83-6a3e-4d30-b813-587f5e341757 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:45.675052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:45.675692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:45.675975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:45.676251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:45.686120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:45.687592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd98362-f1f5-4729-9d8d-b4682a0bd0f9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:45.697097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 725/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:47.797220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabbb2-8710-46c7-9adb-ff0a23e39b15 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:47.797784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabbb2-8710-46c7-9adb-ff0a23e39b15 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:47.797898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befabbb2-8710-46c7-9adb-ff0a23e39b15 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:47.799497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:47.805435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786467-6016-44a6-ba82-65fcd3a25ca8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:47.805859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786467-6016-44a6-ba82-65fcd3a25ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:47.805989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db786467-6016-44a6-ba82-65fcd3a25ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:47.825251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-db786467-6016-44a6-ba82-65fcd3a25ca8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:47.826574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-db786467-6016-44a6-ba82-65fcd3a25ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:47.835636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:47.874102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befabbb2-8710-46c7-9adb-ff0a23e39b15 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:47.885965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befabbb2-8710-46c7-9adb-ff0a23e39b15 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:47.889416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:47.890046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:47.890328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:47.890628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:47.901291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:47.902790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7de4f-1356-4e42-a7fa-0418fc79ae2e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:47.913084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 729/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:50.040117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab86d3-b14a-447f-b988-18b15f958cd9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:50.040702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab86d3-b14a-447f-b988-18b15f958cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:50.040702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab86d3-b14a-447f-b988-18b15f958cd9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:50.042079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:50.047471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37756cb6-197f-4a48-a390-01722dd37fbe None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:50.047917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37756cb6-197f-4a48-a390-01722dd37fbe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:50.048016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37756cb6-197f-4a48-a390-01722dd37fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:50.066920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-37756cb6-197f-4a48-a390-01722dd37fbe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:50.068158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-37756cb6-197f-4a48-a390-01722dd37fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:50.075922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:50.116382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab86d3-b14a-447f-b988-18b15f958cd9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:50.127413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab86d3-b14a-447f-b988-18b15f958cd9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:50.131060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:50.131691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:50.131970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:50.132266 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:50.141217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:50.142691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a494-8984-43ed-a4f2-724e68ecafd9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:50.154781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 728/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:52.311749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872fce-52f5-4c69-a6eb-d9488be15f1a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:52.312351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872fce-52f5-4c69-a6eb-d9488be15f1a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:52.312351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872fce-52f5-4c69-a6eb-d9488be15f1a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:52.313674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:52.319674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bd6a-d348-4c3d-bf94-602089906ae2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:52.320161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bd6a-d348-4c3d-bf94-602089906ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:52.320270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87bd6a-d348-4c3d-bf94-602089906ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:52.326326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=70 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:36:52.339104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fe87bd6a-d348-4c3d-bf94-602089906ae2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:52.340337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fe87bd6a-d348-4c3d-bf94-602089906ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:52.351014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:52.390495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872fce-52f5-4c69-a6eb-d9488be15f1a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:52.396593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=73 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:36:52.402567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872fce-52f5-4c69-a6eb-d9488be15f1a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:52.406170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:52.406842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:52.407127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:52.407401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:52.417504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:52.419035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa4d320-5e81-459c-abd9-620ad5d65bba service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:52.428822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 732/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:54.528334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762d32-034d-4430-82a2-1d75ce9cbf00 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:54.528871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762d32-034d-4430-82a2-1d75ce9cbf00 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:54.528871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762d32-034d-4430-82a2-1d75ce9cbf00 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:54.530289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:54.535566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f34e9e-dbea-4965-82f4-323e75e64d5c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:54.536046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f34e9e-dbea-4965-82f4-323e75e64d5c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:54.536162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f34e9e-dbea-4965-82f4-323e75e64d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:54.554332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-55f34e9e-dbea-4965-82f4-323e75e64d5c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:54.555549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-55f34e9e-dbea-4965-82f4-323e75e64d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:54.563270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:54.601963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762d32-034d-4430-82a2-1d75ce9cbf00 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:54.612902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762d32-034d-4430-82a2-1d75ce9cbf00 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:54.617048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:54.617745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:54.617975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:54.618253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:54.627623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:54.629059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a61d6-18f9-4646-9bfb-4cb1cdf376c9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:54.639098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 731/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:36:56.721829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33d287-665d-4e8f-9f4f-1a789fdee44d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:56.722359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33d287-665d-4e8f-9f4f-1a789fdee44d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:56.722468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33d287-665d-4e8f-9f4f-1a789fdee44d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:56.724675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:56.730115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef6a78-5f49-4458-aa72-8018e4aa8dff None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:56.730655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef6a78-5f49-4458-aa72-8018e4aa8dff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:56.730829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef6a78-5f49-4458-aa72-8018e4aa8dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:56.749835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-92ef6a78-5f49-4458-aa72-8018e4aa8dff None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:56.751068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-92ef6a78-5f49-4458-aa72-8018e4aa8dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:56.759711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:36:56.797391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33d287-665d-4e8f-9f4f-1a789fdee44d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:56.808859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33d287-665d-4e8f-9f4f-1a789fdee44d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:56.812448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:56.813087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:56.813367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:56.813639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:56.823156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:56.824663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61be51-4a67-488e-9420-f03b6ff223b3 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:56.834700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 735/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:36:58.921846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa82f1-a50b-46a5-bc4a-1f383daa4ebd None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:58.922393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa82f1-a50b-46a5-bc4a-1f383daa4ebd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:58.922393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa82f1-a50b-46a5-bc4a-1f383daa4ebd None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:58.923807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:36:58.930544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b28b-ee93-4edc-8c70-7bfe2002476b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:58.931018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b28b-ee93-4edc-8c70-7bfe2002476b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:58.931113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851b28b-ee93-4edc-8c70-7bfe2002476b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:58.950941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6851b28b-ee93-4edc-8c70-7bfe2002476b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:36:58.952136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6851b28b-ee93-4edc-8c70-7bfe2002476b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:36:58.960105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:36:58.997694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa82f1-a50b-46a5-bc4a-1f383daa4ebd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:36:59.009279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa82f1-a50b-46a5-bc4a-1f383daa4ebd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:36:59.012898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:36:59.013540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:36:59.013819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:36:59.014096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:36:59.023582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:36:59.025014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ef3e8-e4c1-4461-9074-a76b7e31fb50 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:36:59.034419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 734/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:08.945825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=17 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:37:09.030210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=17 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:37:20.195248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed13f-ee92-43a1-a14d-b1aeb3c0afb9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:20.195886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed13f-ee92-43a1-a14d-b1aeb3c0afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:20.195886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ed13f-ee92-43a1-a14d-b1aeb3c0afb9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:20.197166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 737/1471] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:20.202568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d939-20c9-425b-96d5-b5375cd62dad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:20.203028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d939-20c9-425b-96d5-b5375cd62dad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:20.203138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d939-20c9-425b-96d5-b5375cd62dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:20.221662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0d59d939-20c9-425b-96d5-b5375cd62dad None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:20.223018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d59d939-20c9-425b-96d5-b5375cd62dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:20.230995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 735/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:20.279477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ab75-3a75-4704-bf63-ed2ee9dc274d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:20.279764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ab75-3a75-4704-bf63-ed2ee9dc274d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:20.280041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ab75-3a75-4704-bf63-ed2ee9dc274d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:20.281402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 738/1473] 10.222.0.23 () {54 vars in 962 bytes} [Fri Jun 24 14:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:37:20.286403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e258e-0a0c-4da7-834e-2a1d8ead0747 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:20.286719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e258e-0a0c-4da7-834e-2a1d8ead0747 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:20.286999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e258e-0a0c-4da7-834e-2a1d8ead0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:20.305747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-388e258e-0a0c-4da7-834e-2a1d8ead0747 None None] MFA Rules not processed for user `bb628c1d9c854cbca433d36d143374c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:20.306984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-388e258e-0a0c-4da7-834e-2a1d8ead0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:20.313416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 736/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Jun 24 14:37:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:37:20.321781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ab75-3a75-4704-bf63-ed2ee9dc274d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:20.332548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ab75-3a75-4704-bf63-ed2ee9dc274d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:20.336221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:20.336852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:20.337130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:20.337399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:20.346537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:20.347973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc2972-c3bb-4649-bedb-896232479a00 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:20.357524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 739/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:22.603334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816069-b152-4935-a6a2-aa26254966a8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:22.603914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816069-b152-4935-a6a2-aa26254966a8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:22.603914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816069-b152-4935-a6a2-aa26254966a8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:22.605313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 737/1476] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:22.610669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24158-b4d1-4f92-a424-d416193120d1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:22.611094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24158-b4d1-4f92-a424-d416193120d1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:22.611208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a24158-b4d1-4f92-a424-d416193120d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:22.629328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d7a24158-b4d1-4f92-a424-d416193120d1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:22.630744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d7a24158-b4d1-4f92-a424-d416193120d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:22.638977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 740/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:22.664374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816069-b152-4935-a6a2-aa26254966a8 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:22.676925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816069-b152-4935-a6a2-aa26254966a8 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:22.681370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:22.682032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:22.682310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:22.682779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:22.683528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:22.685324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:37:22.685324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:37:22.685324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:37:22.686448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c55b44-712c-4f49-9135-0b725606858a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:22.707891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 738/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Fri Jun 24 14:37:22 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:37:24.813232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89947dd6-b1de-4233-b7e8-c29c641e5a10 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:24.813816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89947dd6-b1de-4233-b7e8-c29c641e5a10 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:24.813816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89947dd6-b1de-4233-b7e8-c29c641e5a10 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:24.815231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 741/1479] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:24.822835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893267e-d0ef-48a5-b2f8-13777f4682d2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:24.823361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893267e-d0ef-48a5-b2f8-13777f4682d2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:24.823361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893267e-d0ef-48a5-b2f8-13777f4682d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:24.841342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b893267e-d0ef-48a5-b2f8-13777f4682d2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:24.842573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b893267e-d0ef-48a5-b2f8-13777f4682d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:24.851777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 739/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:24.897208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89947dd6-b1de-4233-b7e8-c29c641e5a10 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:24.912282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89947dd6-b1de-4233-b7e8-c29c641e5a10 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:24.917997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:24.921581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:24.921581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:24.921581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:24.928783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:24.930448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f14996-e25a-4aa7-a7f6-bef58581e89f service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:24.940218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 742/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:27.240903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac089-b760-4f55-b887-702c2e131ba0 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:27.241519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac089-b760-4f55-b887-702c2e131ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:27.241519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac089-b760-4f55-b887-702c2e131ba0 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:27.242946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 740/1482] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:27.249429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dca46-e018-4abd-839f-0613b8d5ca07 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:27.249859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dca46-e018-4abd-839f-0613b8d5ca07 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:27.249964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dca46-e018-4abd-839f-0613b8d5ca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:27.270762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2d3dca46-e018-4abd-839f-0613b8d5ca07 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:27.272012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d3dca46-e018-4abd-839f-0613b8d5ca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:27.280214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 743/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:27.319645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ac089-b760-4f55-b887-702c2e131ba0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:27.333006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ac089-b760-4f55-b887-702c2e131ba0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:27.337098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:27.337763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:27.338092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:27.338373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:27.348341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:27.351236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4ac6e-f1fa-4b48-bbb6-190002ab99c7 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:27.361629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 741/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:37:29.466063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f2067-b09c-4a39-9dec-bbfbd15c8b83 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:29.466700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f2067-b09c-4a39-9dec-bbfbd15c8b83 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:29.466700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f2067-b09c-4a39-9dec-bbfbd15c8b83 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:29.468040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 744/1485] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:29.473984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66598-fcf5-4cc6-bc45-3914eaf319ad None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:29.474385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66598-fcf5-4cc6-bc45-3914eaf319ad None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:29.474563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66598-fcf5-4cc6-bc45-3914eaf319ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:29.492730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-84b66598-fcf5-4cc6-bc45-3914eaf319ad None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:29.493941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-84b66598-fcf5-4cc6-bc45-3914eaf319ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:29.501992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 742/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:29.542127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f2067-b09c-4a39-9dec-bbfbd15c8b83 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:29.554902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f2067-b09c-4a39-9dec-bbfbd15c8b83 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:29.558380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:29.559885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:29.559885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:29.559885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:29.569907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:29.571361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421edbc7-8166-4600-831f-02af8a3afab8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:29.580818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 745/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:31.835131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961f1d-cd1e-4234-98ee-be1844b1f4fe None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:31.835696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961f1d-cd1e-4234-98ee-be1844b1f4fe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:31.835696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39961f1d-cd1e-4234-98ee-be1844b1f4fe None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:31.837039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 743/1488] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:31.842206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaa6b5-ca39-4efd-8f50-b22ae45933b7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:31.842684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaa6b5-ca39-4efd-8f50-b22ae45933b7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:31.842795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaa6b5-ca39-4efd-8f50-b22ae45933b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:31.860383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-abfaa6b5-ca39-4efd-8f50-b22ae45933b7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:31.861572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-abfaa6b5-ca39-4efd-8f50-b22ae45933b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:31.870028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 746/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:37:31.911172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39961f1d-cd1e-4234-98ee-be1844b1f4fe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:31.921966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39961f1d-cd1e-4234-98ee-be1844b1f4fe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:31.926684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:31.927324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:31.927631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:31.927910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:31.937466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:31.938978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741631c-6506-44ff-9381-78599cf3dcb6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:31.949880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 744/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:34.513972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1659-616b-4d20-9ebe-d339cb28ccb6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:34.514567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1659-616b-4d20-9ebe-d339cb28ccb6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:34.514567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a1659-616b-4d20-9ebe-d339cb28ccb6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:34.515951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 747/1491] 10.222.0.23 () {54 vars in 926 bytes} [Fri Jun 24 14:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:37:34.522210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd318a3-83ea-4d16-8ed3-8a25f0e8e283 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:34.522671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd318a3-83ea-4d16-8ed3-8a25f0e8e283 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:34.522780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd318a3-83ea-4d16-8ed3-8a25f0e8e283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:34.542217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ffd318a3-83ea-4d16-8ed3-8a25f0e8e283 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:37:34.543464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ffd318a3-83ea-4d16-8ed3-8a25f0e8e283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:37:34.552013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 745/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Fri Jun 24 14:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:37:34.593388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a1659-616b-4d20-9ebe-d339cb28ccb6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:37:34.605889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a1659-616b-4d20-9ebe-d339cb28ccb6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:37:34.609578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:37:34.610226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:37:34.610509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:37:34.610891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:37:34.620959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:37:34.622406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f974957-5557-44a9-85cd-c01d97e52f98 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:37:34.633006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 748/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:37:44.537003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=33 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:37:44.628138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=33 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:39:26.603149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:26.603753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:26.603753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:26.622703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:39:26.623843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:26.632416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 746/1494] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:39:26.642125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:26.642698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:26.642698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:26.662206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:39:26.663582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:26.671527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 749/1495] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:39:26.676325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:26.686923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ae5bd-597b-4c28-b034-940c8c576178 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:26.691439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:26.692089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:26.692370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:26.692705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:26.693431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:39:26.693734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:26.694786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:26.694786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:39:26.694786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:39:26.695666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:26.707404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 747/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:39:26.715658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:26.725867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2284c41-49fb-4c18-b22d-6714f1016b6d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:26.730668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:26.731364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:26.731646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:26.731917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:26.732599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:26.734626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:26.734626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:39:26.734626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:39:26.735764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:26.834021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 750/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:39:26 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:39:26.838249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:26.848593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1c5a-c100-42d5-b672-950750760f69 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:26.907248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:26.907843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:26.908125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:26.908401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:26.909104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:26.910155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:26.910155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:39:26.910155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:39:26.912062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:26.976011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 748/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:39:26 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:39:26.984022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:26.994960 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab352e7-0b2a-471b-97d0-08fc9173020c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:26.999524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:27.000184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.000465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.000737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.001462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:27.002484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:27.002484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:39:27.002484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:39:27.003389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:27.016518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 751/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:39:27.020651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:27.031846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8cb813-445c-4e99-90c9-767a215bc83f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:27.037293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:27.037949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.038232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.038529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.039258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:27.040307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:27.040307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:39:27.040307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:39:27.041184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4911089-84f3-4749-9ee6-823a707dbd0c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:27.054539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 749/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:39:27.058742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:27.068783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30dbe73-02be-4379-9f3d-42ecc1d5b2af admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:27.073293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:27.074176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.074455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.074764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] PATH_INFO: `/v3/projects/4ba570a4fc1a4d16a16a57a66c7a6c3a/users/e745052630444960bd8c0f6c51821aa0/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.090529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba570a4fc1a4d16a16a57a66c7a6c3a, user_id=e745052630444960bd8c0f6c51821aa0, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:27.091580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:27.091580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:39:27.091580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:39:27.093501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26255cda-ebc3-4149-b484-d354f5f3ec74 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:27.175214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 752/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:39:27 2022] PUT /identity/v3/projects/4ba570a4fc1a4d16a16a57a66c7a6c3a/users/e745052630444960bd8c0f6c51821aa0/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:39:27.186732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f8f1c-f198-486d-95ec-d2fb1a0e3289 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.187023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f8f1c-f198-486d-95ec-d2fb1a0e3289 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.187301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f8f1c-f198-486d-95ec-d2fb1a0e3289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.215568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ca8f8f1c-f198-486d-95ec-d2fb1a0e3289 None None] MFA Rules not processed for user `e745052630444960bd8c0f6c51821aa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:39:27.216791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ca8f8f1c-f198-486d-95ec-d2fb1a0e3289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:27.288758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 750/1502] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:39:27.302000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f29511-89bd-4778-9731-7b2e96355b1c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.302282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f29511-89bd-4778-9731-7b2e96355b1c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.302919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f29511-89bd-4778-9731-7b2e96355b1c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.303955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 753/1503] 10.222.0.23 () {54 vars in 963 bytes} [Fri Jun 24 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:39:27.308910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d40fe-befe-42ed-90fd-6ecbee58f0ef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.309196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d40fe-befe-42ed-90fd-6ecbee58f0ef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.309511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d40fe-befe-42ed-90fd-6ecbee58f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.329281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-964d40fe-befe-42ed-90fd-6ecbee58f0ef None None] MFA Rules not processed for user `c42ba8788bf2431a96b2087b287d7426`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:39:27.330552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-964d40fe-befe-42ed-90fd-6ecbee58f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:27.376099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:39:27.384191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 751/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Fri Jun 24 14:39:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:39:27.392146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f29511-89bd-4778-9731-7b2e96355b1c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:27.403331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f29511-89bd-4778-9731-7b2e96355b1c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:27.406821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:27.407491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.413976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.413976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.416612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:27.418053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:27.427387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 754/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:39:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:39:27.475914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d0e4-9af4-4ad4-8c05-22b8b7817b52 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.476401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d0e4-9af4-4ad4-8c05-22b8b7817b52 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.476555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788d0e4-9af4-4ad4-8c05-22b8b7817b52 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.477853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 752/1506] 10.222.0.23 () {52 vars in 829 bytes} [Fri Jun 24 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:39:27.481583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:27.549286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12609e25-b1c5-4a93-9392-e5d4eca49a04 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:27.554711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:27.555587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:27.555872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:27.556147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] PATH_INFO: `/v3/users/e745052630444960bd8c0f6c51821aa0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:27.557336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e745052630444960bd8c0f6c51821aa0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:27.558365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:27.558365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:39:27.558365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:39:27.559607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:27.931652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e745052630444960bd8c0f6c51821aa0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:27.951463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e745052630444960bd8c0f6c51821aa0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:28.032076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e745052630444960bd8c0f6c51821aa0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:28.042445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:39:28.046798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e745052630444960bd8c0f6c51821aa0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:28.047518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e745052630444960bd8c0f6c51821aa0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:28.055069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 755/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:39:27 2022] DELETE /identity/v3/users/e745052630444960bd8c0f6c51821aa0 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:39:28.059587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:28.060047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:28.060147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:28.077649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:39:28.078901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:28.139391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 753/1508] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:39:28.149660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:28.151149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:28.210006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db89bbe-c9ab-43df-8a50-878c235e0f15 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:28.213620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:28.214364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:28.214672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:28.214948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:28.225057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:28.226483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358f6f0-7874-4b78-b7e6-5ebe32b3f2f7 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:28.236521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 756/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:39:28.776277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:39:28.777799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:39:28.790373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087f293-b615-4ba7-aea0-93b804d608d5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:39:28.795277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:39:28.796152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:39:28.796428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:39:28.796698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] PATH_INFO: `/v3/projects/4ba570a4fc1a4d16a16a57a66c7a6c3a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:39:28.798070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba570a4fc1a4d16a16a57a66c7a6c3a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:39:28.799112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:39:28.799112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:39:28.799112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:39:28.800321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:39:29.243902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba570a4fc1a4d16a16a57a66c7a6c3a', 'request_id': 'req-d63f1223-6437-495d-9845-643c2cedb77d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:29.258707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba570a4fc1a4d16a16a57a66c7a6c3a', 'request_id': 'req-d63f1223-6437-495d-9845-643c2cedb77d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:29.403776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba570a4fc1a4d16a16a57a66c7a6c3a', 'request_id': 'req-d63f1223-6437-495d-9845-643c2cedb77d'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:29.415763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] The token cache is being invalidate because project 4ba570a4fc1a4d16a16a57a66c7a6c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:39:29.416056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:29.416809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d63f1223-6437-495d-9845-643c2cedb77d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:39:29.418384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 754/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:39:28 2022] DELETE /identity/v3/projects/4ba570a4fc1a4d16a16a57a66c7a6c3a => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:39:38.232119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=29 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:39:39.412488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:41:13.344214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:13.344862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:13.344862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:13.346217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:13.351491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e84b-c85c-44a5-90d8-6f4d40cf9e7c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:13.351959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e84b-c85c-44a5-90d8-6f4d40cf9e7c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:13.352088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1599e84b-c85c-44a5-90d8-6f4d40cf9e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:13.370385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1599e84b-c85c-44a5-90d8-6f4d40cf9e7c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:13.371666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1599e84b-c85c-44a5-90d8-6f4d40cf9e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:13.430660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:13.587147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:13.588865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:13.670778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffdb10-f3fc-4f93-a38a-b738c8f39f25 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:13.676483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:13.678966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:13.678966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:13.678966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:13.686867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:13.688576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd3f56-a858-4646-9b5a-fd300bedafed service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:13.698428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 758/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:14.536548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6626-f959-4852-ae4a-0aff1bf7affb None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:14.537107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6626-f959-4852-ae4a-0aff1bf7affb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:14.537107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf6626-f959-4852-ae4a-0aff1bf7affb None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:14.538497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:14.543557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fb6a3-9f32-49b1-b690-d784d31e4efa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:14.544000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fb6a3-9f32-49b1-b690-d784d31e4efa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:14.544118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fb6a3-9f32-49b1-b690-d784d31e4efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:14.562587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c38fb6a3-9f32-49b1-b690-d784d31e4efa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:14.563808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c38fb6a3-9f32-49b1-b690-d784d31e4efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:14.571628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:14.733205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf6626-f959-4852-ae4a-0aff1bf7affb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:14.747864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf6626-f959-4852-ae4a-0aff1bf7affb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:14.753439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:14.754068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:14.754351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:14.754898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:14.763129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:14.765254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c52e553-70b8-4c37-bc4d-1002033383ff service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:14.774739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 757/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:15.336646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee742-d59e-456d-8e1d-01cce28e105a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:15.336930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee742-d59e-456d-8e1d-01cce28e105a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:15.337208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee742-d59e-456d-8e1d-01cce28e105a None None] PATH_INFO: `/v3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:15.338537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 760/1517] 10.222.0.43 () {54 vars in 925 bytes} [Fri Jun 24 14:41:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:41:15.344723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b206b2-b451-4845-a00c-528181ac6810 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:15.345013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b206b2-b451-4845-a00c-528181ac6810 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:15.345293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b206b2-b451-4845-a00c-528181ac6810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:15.365327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-71b206b2-b451-4845-a00c-528181ac6810 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:15.366563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71b206b2-b451-4845-a00c-528181ac6810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:15.429707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 758/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 14:41:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:24.557149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=37 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:41:24.781292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe23c6-9551-4213-8422-8788910f4697 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:24.781478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe23c6-9551-4213-8422-8788910f4697 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:24.781758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe23c6-9551-4213-8422-8788910f4697 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:24.783175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 761/1519] 10.222.0.23 () {54 vars in 960 bytes} [Fri Jun 24 14:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:24.788070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d57d-50f2-4cab-8e09-e508e0ce7b31 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:24.788522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d57d-50f2-4cab-8e09-e508e0ce7b31 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:24.788617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5434d57d-50f2-4cab-8e09-e508e0ce7b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:24.807670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5434d57d-50f2-4cab-8e09-e508e0ce7b31 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:24.808911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5434d57d-50f2-4cab-8e09-e508e0ce7b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:24.817653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 759/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Fri Jun 24 14:41:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:24.825198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe23c6-9551-4213-8422-8788910f4697 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:24.838541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe23c6-9551-4213-8422-8788910f4697 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:24.844136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:24.844768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:24.845050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:24.845326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:24.853534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:24.855230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500edb38-134b-4609-8721-f972e5240e5a service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:24.865469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 762/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:26.074440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343383-3b9c-4074-bda9-3dc193d1c0fa None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:26.075243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343383-3b9c-4074-bda9-3dc193d1c0fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:26.075243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343383-3b9c-4074-bda9-3dc193d1c0fa None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:26.077445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:26.082491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dbb98-fc80-4067-811a-8693c9af0599 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:26.082906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dbb98-fc80-4067-811a-8693c9af0599 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:26.083121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dbb98-fc80-4067-811a-8693c9af0599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:26.102375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f69dbb98-fc80-4067-811a-8693c9af0599 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:26.103628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f69dbb98-fc80-4067-811a-8693c9af0599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:26.112379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:41:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:26.267363 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343383-3b9c-4074-bda9-3dc193d1c0fa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:26.281566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343383-3b9c-4074-bda9-3dc193d1c0fa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:26.286975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:26.287632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:26.287918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:26.288195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:26.296940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:26.298574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fb592-004c-4e48-8760-2a8dacd3acd5 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:26.308073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 761/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:27.254459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01108a-b749-45b8-9057-e11389efa518 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:27.254459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01108a-b749-45b8-9057-e11389efa518 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:27.254459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01108a-b749-45b8-9057-e11389efa518 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:27.254459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 14:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132727-a0bb-4b91-90fa-729fba8bcff8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132727-a0bb-4b91-90fa-729fba8bcff8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132727-a0bb-4b91-90fa-729fba8bcff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-24132727-a0bb-4b91-90fa-729fba8bcff8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24132727-a0bb-4b91-90fa-729fba8bcff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:27.256082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 14:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:27.352258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01108a-b749-45b8-9057-e11389efa518 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:27.367395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01108a-b749-45b8-9057-e11389efa518 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:27.373656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:27.374309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:27.374614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:27.374890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:27.383830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:27.385475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d210-459c-47f4-9142-2c91d9e1fc42 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:27.395228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 765/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:31.415299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:31.415887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:31.415887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:31.417235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:31.423508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be3c83-ae25-4b1d-b4d5-8f513de88682 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:31.423975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be3c83-ae25-4b1d-b4d5-8f513de88682 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:31.424135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be3c83-ae25-4b1d-b4d5-8f513de88682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:31.441998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-65be3c83-ae25-4b1d-b4d5-8f513de88682 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:31.443281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65be3c83-ae25-4b1d-b4d5-8f513de88682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:31.451369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:31.489034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:31.490763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:31.549972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20eac0-e3b5-47e9-9fb0-bca9ba3d9239 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:31.553783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:31.554429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:31.554774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:31.555056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:31.564696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:31.567599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f77e2-dec0-41fd-b1b9-c12b4f4d75ba service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:31.571134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:41:31.575949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 764/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:33.736698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ca02-4d73-498a-95a2-35b04624ae6e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:33.737236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ca02-4d73-498a-95a2-35b04624ae6e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:33.737676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547ca02-4d73-498a-95a2-35b04624ae6e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:33.739594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:33.745906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f807b7-8517-4e37-8530-49d12e904e7e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:33.745906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f807b7-8517-4e37-8530-49d12e904e7e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:33.745906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f807b7-8517-4e37-8530-49d12e904e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:33.765970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c5f807b7-8517-4e37-8530-49d12e904e7e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:33.767432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5f807b7-8517-4e37-8530-49d12e904e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:33.775572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:33.812759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547ca02-4d73-498a-95a2-35b04624ae6e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:33.823733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547ca02-4d73-498a-95a2-35b04624ae6e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:33.827805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:33.828440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:33.828779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:33.829057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:33.838262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:33.839552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36aec4-5b17-47ca-8c3c-53208de02a75 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:33.849171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 768/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:37.575005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:37.576189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:37.576189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:37.577488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:37.582799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d06d74-2398-4a26-a2e5-fffb77c78f76 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:37.593832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d06d74-2398-4a26-a2e5-fffb77c78f76 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:37.593832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d06d74-2398-4a26-a2e5-fffb77c78f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:37.601439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-64d06d74-2398-4a26-a2e5-fffb77c78f76 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:37.602686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-64d06d74-2398-4a26-a2e5-fffb77c78f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:37.610758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:37.923614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:37.925523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:37.984076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0964bd-460b-4afe-bf87-dbc7afec1a95 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:37.987682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:37.988294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:37.988655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:37.988852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:37.998619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:38.000061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030676eb-c629-4a57-9ba8-9ed0f7029f88 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:38.009872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 767/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:40.091287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:40.091860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:40.091860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:40.093305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 14:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:41:40.099129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499002-a6c8-4880-af71-81b6fb21d9ab None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:40.099561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499002-a6c8-4880-af71-81b6fb21d9ab None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:40.099675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2499002-a6c8-4880-af71-81b6fb21d9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:40.119799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c2499002-a6c8-4880-af71-81b6fb21d9ab None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:41:40.121004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2499002-a6c8-4880-af71-81b6fb21d9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:40.131061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 14:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:41:40.433810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:41:40.436331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:41:40.450473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d064519-b029-441d-b941-2ed66b448f8e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:41:40.454264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:41:40.454948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:41:40.455263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:41:40.455541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:41:40.464646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:41:40.466323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb29c6-6fd4-48e8-b534-70cf7db0be52 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:41:40.476251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 771/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:41:50.112981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:41:50.471512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=33 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:11.295805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.296494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.296494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.323741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.324955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.333621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 769/1540] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.344051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be382-3c77-4c98-92de-873e6458a240 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.344630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be382-3c77-4c98-92de-873e6458a240 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.344630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be382-3c77-4c98-92de-873e6458a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.379363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-093be382-3c77-4c98-92de-873e6458a240 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.380664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-093be382-3c77-4c98-92de-873e6458a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.401648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 772/1541] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.406204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:11.421425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca5201-00cb-4f82-a7c9-12d7849dc2bc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:11.432430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:11.433425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.433893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.434333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.435259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:11.435739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:11.437887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:11.458154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 770/1542] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:11.466752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093be382-3c77-4c98-92de-873e6458a240 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:11.484685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093be382-3c77-4c98-92de-873e6458a240 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:11.498956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:11.499651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.499934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.500210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.500915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:11.503148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb91dbf-ba6f-484a-99f5-1b555d892710 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:11.615317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 773/1543] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:11.619659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:11.634540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c097381-f772-41a7-a506-1369d0ee264e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:11.727268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:11.728261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.728726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.729165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.730035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:11.732412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9bd8ee-3f62-4dfb-9c08-ceb2da1f378a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:11.763905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823d0e4-af4c-4af5-8736-ed336432cb93 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.764586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823d0e4-af4c-4af5-8736-ed336432cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.765101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823d0e4-af4c-4af5-8736-ed336432cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.797198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c823d0e4-af4c-4af5-8736-ed336432cb93 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.801817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c823d0e4-af4c-4af5-8736-ed336432cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.812376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 774/1544] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.814975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e837-2b3d-4857-a5dc-404fd34ab66a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.815308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e837-2b3d-4857-a5dc-404fd34ab66a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.815535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0e837-2b3d-4857-a5dc-404fd34ab66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.848635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0cc0e837-2b3d-4857-a5dc-404fd34ab66a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.849878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0cc0e837-2b3d-4857-a5dc-404fd34ab66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.863149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 775/1545] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.864578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.865101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.865486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.883141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 771/1546] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:11.887749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47d3f-9f6e-4498-8043-2b3b0f69bb61 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.887749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47d3f-9f6e-4498-8043-2b3b0f69bb61 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.887749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47d3f-9f6e-4498-8043-2b3b0f69bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.899643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.901201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.925216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 776/1547] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.926414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:11.926561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-adb47d3f-9f6e-4498-8043-2b3b0f69bb61 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.927912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-adb47d3f-9f6e-4498-8043-2b3b0f69bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.948021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ecac6-0dbe-4172-b33e-4bda3a399a17 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:11.956253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 772/1548] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:11.957574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eaf9-86de-4bb6-9be2-a4c6573762fc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.957865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eaf9-86de-4bb6-9be2-a4c6573762fc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.958139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003eaf9-86de-4bb6-9be2-a4c6573762fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.966320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:11.967253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:11.967539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:11.967825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:11.968969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:11.971022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:11.988117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2003eaf9-86de-4bb6-9be2-a4c6573762fc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:11.989332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2003eaf9-86de-4bb6-9be2-a4c6573762fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:11.989779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 777/1549] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:11.990427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.003507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 773/1550] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.003845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e770f-756f-47c3-a33c-9069a0b977de admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.004824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c843e2-f158-49b0-b704-93bc76b9768d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.005116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c843e2-f158-49b0-b704-93bc76b9768d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.005398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c843e2-f158-49b0-b704-93bc76b9768d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.010086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.010794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.011188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.011586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.012330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.012641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.013669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:12.013669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:43:12.013669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:43:12.014603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0e919-9391-4577-bf68-6fa50c224f55 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.026104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f4c843e2-f158-49b0-b704-93bc76b9768d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.027366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f4c843e2-f158-49b0-b704-93bc76b9768d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.029451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 778/1551] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.030939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.031330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.031992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.043346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 774/1552] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.044637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.047362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.047859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.054118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.055387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.066644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 779/1553] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.066644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.073932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.075343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.077893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33853f-7732-4c51-a5fe-b8868d9622fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.082599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.083301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.083585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.083792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 775/1554] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.085111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.088453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.089317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.090999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.096059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da926937-4ee5-449c-9ce7-eccfb5675e4f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.101637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.102364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 780/1555] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:12.102495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.102621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.102869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.103097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.103700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.104017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.105625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597008d7-b0ca-4b01-8183-8d86a30c4bb1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.112767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cb5a8-d8da-41ee-b8ea-12f56e044434 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.117588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.117735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 776/1556] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.118175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.118460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.118786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.118916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.119151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.119428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.119549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.121392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.137435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.138574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.146400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 777/1557] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.150727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.157789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48b4da-b3cd-435d-bf46-f54b4f65d292 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.163466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.164122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.164404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.164679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.165398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.165708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.167370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4512fb-0cee-4e1d-aff8-499ac2ba0925 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.178887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 778/1558] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.180111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3f5d1-a0e5-403e-80fe-7e348d41d417 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.180398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3f5d1-a0e5-403e-80fe-7e348d41d417 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.180672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3f5d1-a0e5-403e-80fe-7e348d41d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.199828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-86c3f5d1-a0e5-403e-80fe-7e348d41d417 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.201145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86c3f5d1-a0e5-403e-80fe-7e348d41d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.209228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 779/1559] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.211527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fce09d-9b71-4b22-a5aa-82597862bd7a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.211814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fce09d-9b71-4b22-a5aa-82597862bd7a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.212093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fce09d-9b71-4b22-a5aa-82597862bd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.229950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-21fce09d-9b71-4b22-a5aa-82597862bd7a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.231176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-21fce09d-9b71-4b22-a5aa-82597862bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.239087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 780/1560] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.244836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 781/1561] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:12.245925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.248703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.254015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.254015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.268913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.270120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.310720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb977409-cad9-4eb6-80cb-7774a2d78389 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.315780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.318329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.318329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.318329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] PATH_INFO: `/v3/projects/fdf303d2dcae43b0ba3bb833111b1a9a/users/0bfcf44f04d04904a12c51d3b25c6fc3/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.323963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:12.333308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 781/1562] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.338003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.347978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76072205-2f99-49df-8eab-fffd86330ea5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.353556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.354258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.354764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.355055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.355775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.356845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:12.356845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:43:12.356845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:43:12.358090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed9b7b-9f3d-447d-9c9c-f3bd19515fc2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.454089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf303d2dcae43b0ba3bb833111b1a9a, user_id=0bfcf44f04d04904a12c51d3b25c6fc3, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.456794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.472731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 782/1563] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:12.474174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.474465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.474775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.493233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.494458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.554644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 783/1564] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.555576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.560484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 782/1565] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:12 2022] PUT /identity/v3/projects/fdf303d2dcae43b0ba3bb833111b1a9a/users/0bfcf44f04d04904a12c51d3b25c6fc3/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:12.561406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.570725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7f392-a87b-4905-bc78-8549b99fa569 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.595690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:12.625401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2cf47-a6fb-4ecb-9a31-8c2150845c3a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.632243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.633287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.633633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.634458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.635298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.635617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.637166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.637301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.637827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.638109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.638385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.639095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.641003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f65bf-9daa-40e7-bb4b-7f8d8d9e0172 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.649331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 783/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.650183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.659888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b81bd-ee02-421a-87ab-ac4a68a158f3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.665130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.665773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.666058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.666329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.667050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.668100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:12.668100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:43:12.668100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:43:12.669312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.784820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 784/1567] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/projects => generated 393 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:12.784820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.784820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.784820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.802835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.804673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:12.810481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 784/1568] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:12.811332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.823782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1a447-792d-4aa0-a236-2aa5e50c83ea admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.866326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 785/1569] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:12.867250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.877276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:12.883247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93982-f461-44a6-8e85-5f23c4e1aed9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.886844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.887559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.887885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.888159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.888880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.888983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.889184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.889638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.889918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.890190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.890833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.890945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.892904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.902906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 785/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.905949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:12.920322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045146c-e6ce-495b-ab0d-5be5512cde90 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:12.924839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:12.925489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.925771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.926045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.926971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:12.927276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:12.929364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d3e49-4e81-444e-a372-6adcfad11068 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:12.941580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 786/1571] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:12.942992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:12.943280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:12.943559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:12.969871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] MFA Rules not processed for user `0bfcf44f04d04904a12c51d3b25c6fc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:12.971094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:13.009234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 786/1572] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.010077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.020113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee4b20-d9b5-466d-a3dd-54f9517e8e78 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.025282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.025927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.026209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.026483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.027216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.029126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.039272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 787/1573] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:13.040239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.050627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6f0a2-4b08-4842-ab88-104408f31e02 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.056033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.056691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.056976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.057249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.057932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.059902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.261737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 787/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.262592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.267002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:13.272202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 788/1575] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.273794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.283421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2501966-d9ec-4d88-96fa-622301bec976 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.327543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8f8a8-cb31-469c-b462-b64e000d8d79 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.332205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.332979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.333267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.333538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.334846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.337133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.344051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.344877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.345230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.345713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.346750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:13.347158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.347356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 788/1576] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:13.347356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.351256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.358681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff947e2-2480-4f5b-b995-47d90676f19c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.363153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 789/1577] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:13.363918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.368289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.368966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.369253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.369533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.370307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.372538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.375930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0080403-3493-4952-8cdf-88faef75f07b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.380661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.381860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.382185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.382471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.383201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.385137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.525344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 789/1578] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.525694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 790/1579] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.526305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.527030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.592214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2882d7-5622-4d21-882d-022e4e3a7d00 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.594699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c0bcf-4370-4b22-856d-e373ce09891f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.596911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.597589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.597873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.598151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.598981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.599381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.600678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.600961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.601233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.601346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.601964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.603976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.611629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 790/1580] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:13.612373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.623527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e2d9f-9a7f-409b-93ae-7333e948d395 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.628808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.629469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.629751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.630030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.630825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.632780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.811121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 791/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/projects => generated 381 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.813335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.843260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 791/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.847786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.854757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6c3e2-acf5-4de8-af3d-fb6712328ec9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.866329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:13.873416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:13.877152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c534a-9da4-49f8-821a-f9c8fc1c35e8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.881770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.883037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.883529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.883831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.884559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.886153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.896501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 792/1583] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:13.897000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.908190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3f312-2eaa-4b28-9b7f-074b12260a03 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.912822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.914619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.914619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.914619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.915431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.917437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.918071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.918857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.919149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.919429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.920198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.921859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.932801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 792/1584] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:13.933679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:13.943479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a8f18-fd23-43e2-81ba-641adb43894a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:13.949946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:13.950705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:13.950994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:13.951273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:13.951982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:13.953973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:13.993085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 793/1585] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:13.993890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.056227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004de8e-5f7e-4352-931d-72b10f5e36f3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.060797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.061507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.061728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.061997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.062704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.064654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.073176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 793/1586] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:14.074039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.084443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d11f0-e731-4d13-8685-b12cab8cbaad admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.089625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.105522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 794/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.106094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.115137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d6e96-ee23-4113-81bf-1dd81a544a61 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.120751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.121402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.121937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.122213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.122927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.124822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.199284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 794/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:13 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:14.200270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.209864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10891dd-5d52-4d62-a312-45b927dad63b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.214474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.215140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.215421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.215693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.216411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.218000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.228278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 795/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.229315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.233589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 795/1590] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:14.233589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.239378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e943c-00c5-4445-9449-b93132a9fa5c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.244827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.245299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893f75-8cc5-4060-bb2f-ffa10a161977 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.245484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.245763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.246034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.247118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.248696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.252359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.253257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.253532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.253806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] PATH_INFO: `/v3/projects/59f8d8f062af47dda1c3e63327c83245/users/56cc2c6c31dd4970b01fdf23fa6eda0d/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.260841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 796/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.261666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.271162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59f8d8f062af47dda1c3e63327c83245, user_id=56cc2c6c31dd4970b01fdf23fa6eda0d, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.271325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc546-2405-411c-a4f1-28800069a0b3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.272216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:14.272216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:43:14.272216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:43:14.274345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.276300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.277021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.277366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.277639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.278312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.280421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.345669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 796/1592] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:14 2022] PUT /identity/v3/projects/59f8d8f062af47dda1c3e63327c83245/users/56cc2c6c31dd4970b01fdf23fa6eda0d/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:14.346422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.376371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 797/1593] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:14.377447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.390136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4705b3-19ad-4efd-8582-81026c45925e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.408729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03172c1f-872f-4964-a3f3-fe539113a26b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.415067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.415739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.416021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.416293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.417017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.418722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.428979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 797/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.430458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.439784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c538e-f0cf-4a8c-bb43-93ec06af475e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.444305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.445024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.445300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.445750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.446549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.448084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.448464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.449499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.449783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.450340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] PATH_INFO: `/v3/projects/1cc37049272e453591997ede21fd4077/users/fa21b2f9801344d084331a40a9f65a28/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.458042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 798/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.458874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.465181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc37049272e453591997ede21fd4077, user_id=fa21b2f9801344d084331a40a9f65a28, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.467892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72270da8-1310-4531-8553-452dec5df542 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.468718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311d0cf-7a7a-4b6d-9e35-87c081570848 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.473298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.473938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.474215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.474485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.475217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.477515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.487386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 799/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.492223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.498322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e425f4b-57f4-4324-8cb1-daf4de8693e3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.503063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.503791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.504113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.504386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.505090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.506686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.517034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 800/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.517866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.523702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:14.528805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc07c6a8-6201-4c84-9116-41ce05a54a37 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.533972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.534872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.535158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.535436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] PATH_INFO: `/v3/projects/0abdd49efea54add810ad0668df4a899/users/174b3629771b4044bb7b1cee523225d1/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.551917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0abdd49efea54add810ad0668df4a899, user_id=174b3629771b4044bb7b1cee523225d1, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.555406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.568362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 798/1598] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:14 2022] PUT /identity/v3/projects/1cc37049272e453591997ede21fd4077/users/fa21b2f9801344d084331a40a9f65a28/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:14.569929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.570409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.570688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.577657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:14.596600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] MFA Rules not processed for user `56cc2c6c31dd4970b01fdf23fa6eda0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:14.597738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:14.660772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 799/1599] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:14.661751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.671765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 801/1600] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:14 2022] PUT /identity/v3/projects/0abdd49efea54add810ad0668df4a899/users/174b3629771b4044bb7b1cee523225d1/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:14.672762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.727806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd699fc2-4649-4547-a777-cc236b8b7904 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.732433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.733100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.733383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.733658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.734265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823e98-c521-42e5-89ba-6115f362c0c7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.734386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.736035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.738963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.739847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.740188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.740700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.742118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.744317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.748996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 800/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.748996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.755241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 802/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.756167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.758038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eab347-87ba-4c6d-85e5-2dde043d7bc7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.762720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.763372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.763649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.763931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.764708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.766255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735614f-5f17-49df-9a97-54d4651580ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.766501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.771047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.771920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.772202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.772476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] PATH_INFO: `/v3/projects/ac4f626daff54f49a6fee1d94fd991d8/users/24999de5b283456fb3505d834c84133e/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.778679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 801/1603] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:14.779739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.787941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4f626daff54f49a6fee1d94fd991d8, user_id=24999de5b283456fb3505d834c84133e, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.790305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179a56a-db1b-4dcd-977b-efe99f6c13e5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.791597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95997558-2d55-4842-bc89-45694b03bca7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.794953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.795963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.796245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.796519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] PATH_INFO: `/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a/users/e2a11dedb3634ac895e3daa3e3f92861/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.811744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c1d13b06324d9f902d43a9bc3fb73a, user_id=e2a11dedb3634ac895e3daa3e3f92861, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.814432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.866160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 803/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:14 2022] PUT /identity/v3/projects/ac4f626daff54f49a6fee1d94fd991d8/users/24999de5b283456fb3505d834c84133e/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:14.869834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76ed7-5cf1-448b-a4f5-41cb2a8fbabe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.869834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76ed7-5cf1-448b-a4f5-41cb2a8fbabe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.869834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f76ed7-5cf1-448b-a4f5-41cb2a8fbabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.894197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 802/1605] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:14 2022] PUT /identity/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a/users/e2a11dedb3634ac895e3daa3e3f92861/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:14.894464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-63f76ed7-5cf1-448b-a4f5-41cb2a8fbabe None None] MFA Rules not processed for user `fa21b2f9801344d084331a40a9f65a28`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:14.895189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:14.899279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-63f76ed7-5cf1-448b-a4f5-41cb2a8fbabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:14.963649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 804/1606] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:14.964373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:14.964932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.965216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.965553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.969869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfda5c-af9d-49fd-84a6-5bc7ec22cc51 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:14.977582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:14.979904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:14.979904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:14.979904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:14.979904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:14.981396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:14.984301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:14.985496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:14.996356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 805/1607] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:14.996356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.004123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66469970-f6b9-4de8-8aa1-c5a6da2c40f1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.009196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.009856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.013940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.013940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.013940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.013940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.023428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 806/1608] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:15.023428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.034420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70a462-89d2-485b-81f2-19a0dc47f07f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.040257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.042622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.042622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.042622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] PATH_INFO: `/v3/projects/132c0bbdee9e46388793b33cda9ee10c/users/ea0f85ae8bd942338bfcda177b6864a6/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.056372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=132c0bbdee9e46388793b33cda9ee10c, user_id=ea0f85ae8bd942338bfcda177b6864a6, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.059061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f587b91b-d616-4ea0-8a55-8e0073cec943 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.072615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 803/1609] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:14 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:15.111210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.139797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f02df1-b910-4bc5-86be-3323f6017a2a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.139797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.140488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.140765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.141036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] PATH_INFO: `/v3/projects/9ec6517616c94b799e70612816bf9d76/users/a5133bd63bd74551bcecc86dac5ca3d6/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.157041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec6517616c94b799e70612816bf9d76, user_id=a5133bd63bd74551bcecc86dac5ca3d6, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.159773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.192544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:15.198005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 807/1610] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:15 2022] PUT /identity/v3/projects/132c0bbdee9e46388793b33cda9ee10c/users/ea0f85ae8bd942338bfcda177b6864a6/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:15.199286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.199570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.199846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.216919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:15.218226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:15.267469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 804/1611] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:15 2022] PUT /identity/v3/projects/9ec6517616c94b799e70612816bf9d76/users/a5133bd63bd74551bcecc86dac5ca3d6/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:15.268835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.280754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 808/1612] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:15.281808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.334289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a19d52-fff4-4059-9963-7576479fcf39 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.339758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.340439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.340719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.340995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.341722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.343353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.343547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee9ada-146e-4d30-b8a9-70717b30c6bc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.356609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 805/1613] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:15.356609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.394572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.395140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.395420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.395693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.416465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b2917-4643-40ef-a162-ada08d5b6bf5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.420197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.420822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.421100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.421373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.430466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.431924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.441672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 806/1614] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:15.442454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.447594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:15.453118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9806f6a9-573e-43e3-bda1-646d8724ce86 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.454972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.456409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.458093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.459084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.467330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.467330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] PATH_INFO: `/v3/projects/9c875d2a5043420e88f676fbc8cc25a5/users/d1e2e6120385486a9a490b97cc1f67d5/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.467570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 809/1615] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:15.467570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.476231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c875d2a5043420e88f676fbc8cc25a5, user_id=d1e2e6120385486a9a490b97cc1f67d5, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.477789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136c26-3d42-4c31-bd1a-d58835041f0b service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.479006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf557b4b-87ee-424a-8d3d-443a3d544d81 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.483676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.484278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.484580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.484854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.485541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.487609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b69afcf-8cd4-4413-a343-90e360091d16 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.496748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:15.626746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 810/1617] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:15 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:15.628568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.629064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.629561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.633344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 807/1617] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:15 2022] PUT /identity/v3/projects/9c875d2a5043420e88f676fbc8cc25a5/users/d1e2e6120385486a9a490b97cc1f67d5/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:15.634904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.635192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.635935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.665807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:15.665807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:15.666158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:15.666158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:15.729111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 811/1618] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:15.730062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.748314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 808/1619] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:15.749127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.765516 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d195e83-9fe6-420e-8297-5c665776b97d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.770303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.780638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.780638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.780638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] PATH_INFO: `/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a/users/e2a11dedb3634ac895e3daa3e3f92861/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.780638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c1d13b06324d9f902d43a9bc3fb73a, user_id=e2a11dedb3634ac895e3daa3e3f92861, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.780638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96ec9c-be1a-435a-ad71-ce5e97a4a7b1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.787124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 809/1620] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:15 2022] PUT /identity/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a/users/e2a11dedb3634ac895e3daa3e3f92861/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:15.788720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.788965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.789244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.793775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487e90c-c699-442a-bf17-c03cd33b7003 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.817258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] MFA Rules not processed for user `d1e2e6120385486a9a490b97cc1f67d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:15.818476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:15.847940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.848744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.849104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.849446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.914172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 810/1621] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:15.918769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.943666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.947348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.954250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:15.960538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 812/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:15.961283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:15.962714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:15.973667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd8d53-e647-4147-87fc-c81286a2c2d7 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:15.979169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:15.979911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:15.980092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:15.980544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:15.995605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:15.997503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:15.999482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:16.005326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b18f6e-6054-41f2-a7c8-7f83d649d001 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.008546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 813/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:16.009459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.010409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.011180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.011399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.011681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.012441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.015139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395f877-2ad8-4016-b70b-b88a6c8af055 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.019743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9acfc84-009d-4463-94a3-fddb37181ca2 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.025045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.026076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.026471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.026880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.027119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 811/1624] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:16.028682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a083-c738-467b-bdce-95d1f7847dd7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.029310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a083-c738-467b-bdce-95d1f7847dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.029310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0a083-c738-467b-bdce-95d1f7847dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.038037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.040627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e0018c-30e3-4ee8-aaba-e1189669190d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.052556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 814/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:16.079901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e003-5516-47e4-a51f-292cffafe2d6 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.079901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e003-5516-47e4-a51f-292cffafe2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.079901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e003-5516-47e4-a51f-292cffafe2d6 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.079901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 815/1626] 10.222.0.23 () {54 vars in 961 bytes} [Fri Jun 24 14:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:16.079901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8e003-5516-47e4-a51f-292cffafe2d6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.080593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f4a0a083-c738-467b-bdce-95d1f7847dd7 None None] MFA Rules not processed for user `e2a11dedb3634ac895e3daa3e3f92861`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:16.080593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f4a0a083-c738-467b-bdce-95d1f7847dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:16.080762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8e003-5516-47e4-a51f-292cffafe2d6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.083468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.084387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.084671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.084953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] PATH_INFO: `/v3/projects/7d4f4ce1fe984e279335d66603bbb39c/users/3a29fb2f733f4971b003bd060d7cc492/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.105096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4f4ce1fe984e279335d66603bbb39c, user_id=3a29fb2f733f4971b003bd060d7cc492, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.108167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.150251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 812/1627] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:16.151594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.151887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.152166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.177149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:16.177149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:16.239027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 816/1628] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:16 2022] PUT /identity/v3/projects/7d4f4ce1fe984e279335d66603bbb39c/users/3a29fb2f733f4971b003bd060d7cc492/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:16.243318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.256175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 813/1629] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Jun 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:16.258653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.274218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873350-32d8-4887-adac-9e98f50ebe18 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.328682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58b4c-b149-45c4-98eb-a97f0b88fe37 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.334926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.336770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.337420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.337868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.349989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.351052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.358637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.358637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.358637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.358637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.372608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 814/1630] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:16.373708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.420286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.422711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.451156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 817/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:16.452123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.465056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211f650-c47a-4238-bd4c-adbe915794a0 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.470091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.471118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.471427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.471912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] PATH_INFO: `/v3/projects/7d4f4ce1fe984e279335d66603bbb39c/users/3a29fb2f733f4971b003bd060d7cc492/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.473942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4f4ce1fe984e279335d66603bbb39c, user_id=3a29fb2f733f4971b003bd060d7cc492, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.480232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f34b4-bd27-48d7-b25a-4f30931ebd7f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.483598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82335930-d149-423c-9151-1572d0aba1f8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.488070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:16.489066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.489432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.489934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.506477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 818/1632] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:16 2022] PUT /identity/v3/projects/7d4f4ce1fe984e279335d66603bbb39c/users/3a29fb2f733f4971b003bd060d7cc492/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:16.521718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:16.523260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:16.523558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:16.584457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:16.584457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:16.608412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 815/1633] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:16.644927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] MFA Rules not processed for user `3a29fb2f733f4971b003bd060d7cc492`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:16.646476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:16.688991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:16.716448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:16.793462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:16.807658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17212270-dbba-41cd-846e-64f4e8385a1c service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:16.819679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 819/1634] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:16.821988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:17.065976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:17.074653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:17.074653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:17.074653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:17.099916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04729d7-5ea2-4ce5-ac64-594572c2ebb5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:17.111393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:17.114995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:17.115489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:17.115940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:17.138884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:17.141591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9caab-0510-438e-a50a-8f8dce943286 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:17.161489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 820/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:17.182123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a078c-b66e-4ce9-b055-93636bb0d2b9 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:17.182426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a078c-b66e-4ce9-b055-93636bb0d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:17.185450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a078c-b66e-4ce9-b055-93636bb0d2b9 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:17.185450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 821/1636] 10.222.0.23 () {54 vars in 912 bytes} [Fri Jun 24 14:43:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:17.195787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:17.196083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:17.196370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:17.251637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:17.258036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:17.259863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:17.283183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 816/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:17.283333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:17.284718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:17.376446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 822/1638] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:17.471024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:17.480817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8fa79-2802-4222-8802-9d4737353702 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:17.484783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:17.485440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:17.485725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:17.485998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:17.498206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:17.499735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490a12f-da08-4cb0-a2fe-d0726267c4f0 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:17.514263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 817/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:17.996578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:18.006072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83685-2bf6-4c84-8428-4dcc5510319e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:18.010994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:18.012540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:18.012540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:18.012540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:18.080180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:18.081922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60a571-ebb5-4784-93cc-01fc24c9d6e5 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:18.093586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 823/1640] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:19.188611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf8cc-d8d7-4f4b-8efe-1fc2ab31e20d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:19.188611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf8cc-d8d7-4f4b-8efe-1fc2ab31e20d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:19.189998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cf8cc-d8d7-4f4b-8efe-1fc2ab31e20d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:19.193315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 818/1641] 10.222.0.23 () {54 vars in 961 bytes} [Fri Jun 24 14:43:19 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:19.200708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fb033-99b2-44a5-b6af-c7c40b1fe8dd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:19.201211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fb033-99b2-44a5-b6af-c7c40b1fe8dd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:19.201307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fb033-99b2-44a5-b6af-c7c40b1fe8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:19.214369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:19.236152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-044fb033-99b2-44a5-b6af-c7c40b1fe8dd None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:19.265454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-044fb033-99b2-44a5-b6af-c7c40b1fe8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:19.265454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 824/1642] 10.222.0.23 () {58 vars in 1058 bytes} [Fri Jun 24 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:19.265799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25074b-6851-4ad2-b394-72def90761e7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:19.265799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25074b-6851-4ad2-b394-72def90761e7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:19.265799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25074b-6851-4ad2-b394-72def90761e7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:19.265799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 819/1643] 10.222.0.23 () {54 vars in 912 bytes} [Fri Jun 24 14:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:19.266059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c66a6-c088-4851-9c8f-7478769db1eb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:19.266059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c66a6-c088-4851-9c8f-7478769db1eb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:19.266059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c66a6-c088-4851-9c8f-7478769db1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:19.291601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-928c66a6-c088-4851-9c8f-7478769db1eb None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:19.292956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-928c66a6-c088-4851-9c8f-7478769db1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:19.311489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 825/1644] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:19.325068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25074b-6851-4ad2-b394-72def90761e7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:19.340651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25074b-6851-4ad2-b394-72def90761e7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:19.345783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:19.346484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:19.346822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:19.347248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:19.371471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:19.373315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94036426-0429-41e5-b72b-6bb5a95fe30f service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:19.385370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 820/1645] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:24.475416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:24.475962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:24.476144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:24.524724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] MFA Rules not processed for user `174b3629771b4044bb7b1cee523225d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:24.526299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:24.607042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 826/1646] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:24.621262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652801-15b1-4991-9951-684435315c93 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:24.622665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652801-15b1-4991-9951-684435315c93 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:24.623137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be652801-15b1-4991-9951-684435315c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:24.652558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-be652801-15b1-4991-9951-684435315c93 None None] MFA Rules not processed for user `174b3629771b4044bb7b1cee523225d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:24.653779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-be652801-15b1-4991-9951-684435315c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:24.666735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 821/1647] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:24.674318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:24.698873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412c5d5-9b31-4fe7-a852-83e160cc6d59 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:24.706078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:24.711611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:24.711611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:24.711611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:24.717567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:24.719411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949b627-c078-43d4-89f8-29582bdf8f76 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:24.731085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 827/1648] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:25.650080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdeeef-6fc9-41a6-8f70-6ea65d5e9483 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:25.650080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdeeef-6fc9-41a6-8f70-6ea65d5e9483 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:25.650080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fdeeef-6fc9-41a6-8f70-6ea65d5e9483 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:25.650080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 822/1649] 10.222.0.23 () {54 vars in 912 bytes} [Fri Jun 24 14:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:25.654919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d2ef-6b53-4c32-8fde-31e793726788 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:25.654919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d2ef-6b53-4c32-8fde-31e793726788 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:25.654919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d2ef-6b53-4c32-8fde-31e793726788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:25.654919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d901d2ef-6b53-4c32-8fde-31e793726788 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:25.654919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d901d2ef-6b53-4c32-8fde-31e793726788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:25.762707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 828/1650] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:43:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:25.784445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdeeef-6fc9-41a6-8f70-6ea65d5e9483 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:25.817723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fdeeef-6fc9-41a6-8f70-6ea65d5e9483 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:25.823743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:25.824803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:25.825254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:25.825686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:25.844942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:25.848001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a642-1cc7-4f0a-a6fb-15f2a49b92db service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:25.871817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 823/1651] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:26.935505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532804d7-fe55-43f6-9443-75936cf8e9a8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:26.935981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532804d7-fe55-43f6-9443-75936cf8e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:26.936067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532804d7-fe55-43f6-9443-75936cf8e9a8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:26.937514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 829/1652] 10.222.0.23 () {54 vars in 960 bytes} [Fri Jun 24 14:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:26.950756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493992b-6b0b-4886-b36b-c16296711a0e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:26.951224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493992b-6b0b-4886-b36b-c16296711a0e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:26.951333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493992b-6b0b-4886-b36b-c16296711a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:26.993041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d493992b-6b0b-4886-b36b-c16296711a0e None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:26.996768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d493992b-6b0b-4886-b36b-c16296711a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:27.027287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:27.027837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:27.028469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:27.056934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] MFA Rules not processed for user `174b3629771b4044bb7b1cee523225d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:27.058380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:27.079818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 830/1653] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:27.156321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 824/1654] 10.222.0.23 () {58 vars in 1057 bytes} [Fri Jun 24 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:27.164837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:27.181605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0905420-7d5a-4b8e-b5f3-61b804bdd758 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:27.234287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:27.253078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:27.253526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:27.254666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:27.258835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:27.281612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:27.283435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:27.317107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 831/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:29.324795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473e74-5a7c-40d6-8560-4c1c1fd5398b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:29.325273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473e74-5a7c-40d6-8560-4c1c1fd5398b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:29.325489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f473e74-5a7c-40d6-8560-4c1c1fd5398b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:29.388891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1f473e74-5a7c-40d6-8560-4c1c1fd5398b None None] MFA Rules not processed for user `ea0f85ae8bd942338bfcda177b6864a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:29.390596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1f473e74-5a7c-40d6-8560-4c1c1fd5398b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:29.552191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 825/1656] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:29.561732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:29.580221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c412c92-0126-48f5-a959-e806b62c2374 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:29.598803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04102097-f049-489d-bb89-935373d65b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:29.600211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102097-f049-489d-bb89-935373d65b11 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:29.605988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102097-f049-489d-bb89-935373d65b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:29.606554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04102097-f049-489d-bb89-935373d65b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:29.636674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04102097-f049-489d-bb89-935373d65b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:29.638463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04102097-f049-489d-bb89-935373d65b11 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:29.661178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 832/1657] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:32.125050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73cb1-c861-40c0-b1d9-00423425842e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:32.126061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73cb1-c861-40c0-b1d9-00423425842e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:32.126361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73cb1-c861-40c0-b1d9-00423425842e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:32.162984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-95f73cb1-c861-40c0-b1d9-00423425842e None None] MFA Rules not processed for user `a5133bd63bd74551bcecc86dac5ca3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:32.164384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-95f73cb1-c861-40c0-b1d9-00423425842e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:32.199790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:32.200289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:32.200480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:32.203806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:32.232754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] MFA Rules not processed for user `24999de5b283456fb3505d834c84133e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:32.234742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:32.299940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:32.313352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 826/1658] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:32.326867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:32.327453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:32.327974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:32.360330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 833/1659] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:32.383822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:32.394440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] MFA Rules not processed for user `a5133bd63bd74551bcecc86dac5ca3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:32.402409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:32.403860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab2dfe-68fc-4040-8587-20ce7371cfaf None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:32.412755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:32.413828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:32.418757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:32.419326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:32.422604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 827/1660] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:32.425537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:32.446016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96eaac-050f-460b-8e40-9d945fba54bd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:32.446296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:32.449231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe00d55-76e4-463c-b955-cd84f96466c2 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:32.450747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:32.452865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:32.453376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:32.453842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:32.472763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:32.507984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 834/1661] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:32.524994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34783a25-ad05-446b-9b40-582032ce9f79 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:32.575063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 828/1662] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:34.756020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:34.756506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:34.756579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:34.795809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] MFA Rules not processed for user `ea0f85ae8bd942338bfcda177b6864a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:34.797309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:34.813500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 835/1663] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:35.509001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718348-226d-4343-9ea1-60a1a8fa5602 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:35.509489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718348-226d-4343-9ea1-60a1a8fa5602 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:35.509561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718348-226d-4343-9ea1-60a1a8fa5602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:35.555737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-af718348-226d-4343-9ea1-60a1a8fa5602 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:35.557341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-af718348-226d-4343-9ea1-60a1a8fa5602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:35.781096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 829/1664] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:36.277475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:36.282646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:36.480008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83af7b-4f38-42be-bfd3-3c3461aeeb35 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:36.503335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:36.505134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:36.505957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:36.506812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:36.543796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:36.555939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bdf3a-c02e-4e33-bc63-f96c478fbc7c service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:36.581207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 836/1665] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:43:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:43:38.881851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:38.883176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:38.884235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:38.918858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] MFA Rules not processed for user `24999de5b283456fb3505d834c84133e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:38.920382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:38.938937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 830/1666] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:39.924761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 837/1667] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:39.926196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:39.935517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048816c0-a649-4da1-8a3a-eb633661f983 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:39.943285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:39.944251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:39.944532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:39.944811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:40.024283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:40.025660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:40.025660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70038131-90a6-42a6-9f27-bce5aaa51239 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:40.025972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e2ae9-2f52-4d64-99cd-d97547f97a05 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:40.025972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:40.025972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:40.025972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:40.026535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:40.026655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 831/1668] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:40.047703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:40.053119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3918b-8a28-4586-83ed-6f98230ce7d9 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:40.088833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 838/1669] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:40.125291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268c8b-68fb-410d-95ea-490e135e5806 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:40.125648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268c8b-68fb-410d-95ea-490e135e5806 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:40.126018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35268c8b-68fb-410d-95ea-490e135e5806 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:40.127481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 832/1670] 10.222.0.23 () {54 vars in 912 bytes} [Fri Jun 24 14:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:40.152528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f52e-86b2-4dab-95d4-d8272b501713 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:40.152821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f52e-86b2-4dab-95d4-d8272b501713 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:40.153147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017f52e-86b2-4dab-95d4-d8272b501713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:40.225681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4017f52e-86b2-4dab-95d4-d8272b501713 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:40.238947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4017f52e-86b2-4dab-95d4-d8272b501713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:40.290204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 839/1671] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:43:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:40.359146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268c8b-68fb-410d-95ea-490e135e5806 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:40.388887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35268c8b-68fb-410d-95ea-490e135e5806 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:40.398464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:40.400197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:40.401760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:40.402853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:40.436761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:40.438778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452b3038-8753-4ee2-99f1-6d58e86b3838 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:40.474666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 833/1672] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:43.574640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a241a-a256-4323-960f-789a812e5e23 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:43.575858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a241a-a256-4323-960f-789a812e5e23 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:43.576295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a241a-a256-4323-960f-789a812e5e23 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:43.578212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 840/1673] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:43:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:43:43.641454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:43.641454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:43.641454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:43.686462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:43.687858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:43.715407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 834/1674] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:43:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:44.124800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:44.125305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a241a-a256-4323-960f-789a812e5e23 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:44.352098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b3696-1ae0-43d0-970f-e6c4077b87c1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:44.371405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:44.376973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:44.376973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:44.376973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] PATH_INFO: `/v3/users/0bfcf44f04d04904a12c51d3b25c6fc3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:44.379230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfcf44f04d04904a12c51d3b25c6fc3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:44.384023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a241a-a256-4323-960f-789a812e5e23 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:44.387751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:44.393607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:44.395045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:44.395459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:44.395745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] PATH_INFO: `/v3/users/d1e2e6120385486a9a490b97cc1f67d5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:44.397020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e2e6120385486a9a490b97cc1f67d5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:44.398065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: WARNING py.warnings [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:44.398065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: warnings.warn(msg) Jun 24 14:43:44.398065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: Jun 24 14:43:44.400735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:44.420477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:44.500600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:44.696116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfcf44f04d04904a12c51d3b25c6fc3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.771284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfcf44f04d04904a12c51d3b25c6fc3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.778123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e2e6120385486a9a490b97cc1f67d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.829416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e2e6120385486a9a490b97cc1f67d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.893866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfcf44f04d04904a12c51d3b25c6fc3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.907474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfcf44f04d04904a12c51d3b25c6fc3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.912766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfcf44f04d04904a12c51d3b25c6fc3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:44.935042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 835/1675] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:44 2022] DELETE /identity/v3/users/0bfcf44f04d04904a12c51d3b25c6fc3 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:44.937266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:44.941831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:44.983796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e2e6120385486a9a490b97cc1f67d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:45.006833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e2e6120385486a9a490b97cc1f67d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:45.019165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-21ba5311-93d9-4543-9c29-5fd568cafa86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e2e6120385486a9a490b97cc1f67d5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:45.054027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 841/1676] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:44 2022] DELETE /identity/v3/users/d1e2e6120385486a9a490b97cc1f67d5 => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:45.061945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344213-7f9d-45eb-9005-2f828fa63c0d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.062193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344213-7f9d-45eb-9005-2f828fa63c0d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.068054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71344213-7f9d-45eb-9005-2f828fa63c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.144288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-71344213-7f9d-45eb-9005-2f828fa63c0d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:45.149964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71344213-7f9d-45eb-9005-2f828fa63c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:45.242735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a83cb-72e2-4fd1-b45a-7473d80301a3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:45.389749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 842/1677] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:45.392970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.393424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.393940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.428898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:45.429633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.430053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.430299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.435258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:45.452413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:45.455941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:45.460076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:45.465979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:45.482253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 836/1678] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:45.483425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:45.484692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 843/1679] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:45.486882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:45.501879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:45.504105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:45.560286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf9ea2-2a1b-44b1-bd71-50602bd5197e service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:45.580962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:45.581975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.585542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.585957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] PATH_INFO: `/v3/users/fa21b2f9801344d084331a40a9f65a28` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.587707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa21b2f9801344d084331a40a9f65a28)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:45.606133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:45.626568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:45.768572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c41be-40e5-40b9-8a32-21ac850f329f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:45.779265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:45.781485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.788740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.801111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.825088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:45.827188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:45.890695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 844/1680] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:45.893186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:45.929869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6e321-e265-4523-b6d3-238ca796facc service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:45.943162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:45.944400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:45.945942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:45.946672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:45.985350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:45.987113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e6fa0-930a-4ef2-971f-b930b08be74f service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:45.995433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa21b2f9801344d084331a40a9f65a28'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:46.029901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 845/1681] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:46.030197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa21b2f9801344d084331a40a9f65a28'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:46.168186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa21b2f9801344d084331a40a9f65a28'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:46.196662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa21b2f9801344d084331a40a9f65a28'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:46.201851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5cf0bdf3-d850-4d72-9ce5-db6c6e8ef98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa21b2f9801344d084331a40a9f65a28'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:46.213558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99c56-e633-4b08-9808-b29387a72ae8 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:46.214041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99c56-e633-4b08-9808-b29387a72ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:46.214462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b99c56-e633-4b08-9808-b29387a72ae8 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:46.216628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 846/1682] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:43:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 14:43:46.260730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 837/1683] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:45 2022] DELETE /identity/v3/users/fa21b2f9801344d084331a40a9f65a28 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:46.262186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:46.267127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:46.267769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:46.343583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:46.345904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:46.392255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:46.461089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:46.461381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:46.461721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:46.508140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 847/1684] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:46.531459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:46.534892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:46.752310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:46.756925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:46.768339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 838/1685] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:43:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:46.938100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7335-5021-4c25-9c83-93b21b664b99 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:46.947354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:46.948276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:46.948754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:46.949098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:46.973018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:46.975305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:47.004115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 848/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:47.216758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:47.225963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:47.298794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e265376-4069-45d5-bad7-00e592ac7dfb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:47.330050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:47.330899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:47.331221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:47.331500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:47.353379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:47.355510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:47.376339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 839/1687] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:48.446646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:48.448201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:48.473335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:48.475483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:48.497087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f459982-8c7a-49fb-ac7e-5afb28a86041 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:48.508150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5e8a5-d3a6-48b2-984a-ed8f3505f0d6 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:48.514724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:48.516247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:48.517103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:48.518382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:48.518937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] PATH_INFO: `/v3/projects/1cc37049272e453591997ede21fd4077` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:48.519363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:48.519650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:48.519929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] PATH_INFO: `/v3/projects/fdf303d2dcae43b0ba3bb833111b1a9a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:48.522332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf303d2dcae43b0ba3bb833111b1a9a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:48.522951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc37049272e453591997ede21fd4077)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:48.523575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 14:43:48.523575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jun 24 14:43:48.523575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: Jun 24 14:43:48.524777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:48.537216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:48.788690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc37049272e453591997ede21fd4077', 'request_id': 'req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.803064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf303d2dcae43b0ba3bb833111b1a9a', 'request_id': 'req-e6be56d0-3125-44fd-a9d6-3294675df1c9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.807723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc37049272e453591997ede21fd4077', 'request_id': 'req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.921273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf303d2dcae43b0ba3bb833111b1a9a', 'request_id': 'req-e6be56d0-3125-44fd-a9d6-3294675df1c9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.931490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:48.949263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf303d2dcae43b0ba3bb833111b1a9a', 'request_id': 'req-e6be56d0-3125-44fd-a9d6-3294675df1c9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.973706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] The token cache is being invalidate because project fdf303d2dcae43b0ba3bb833111b1a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:48.974344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.977000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:48.983846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 840/1688] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:48 2022] DELETE /identity/v3/projects/fdf303d2dcae43b0ba3bb833111b1a9a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:48.985183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:48.991722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:49.046764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc37049272e453591997ede21fd4077', 'request_id': 'req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.069853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] The token cache is being invalidate because project 1cc37049272e453591997ede21fd4077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:49.070160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.072987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7731e6dc-6657-4a66-a2e6-c4ecc8431e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.075807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 849/1689] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:48 2022] DELETE /identity/v3/projects/1cc37049272e453591997ede21fd4077 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:49.080492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378999-95db-498a-ae14-f67b148bf6ae None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:49.087131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378999-95db-498a-ae14-f67b148bf6ae None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:49.087131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5378999-95db-498a-ae14-f67b148bf6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:49.127320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b5378999-95db-498a-ae14-f67b148bf6ae None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:49.129960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b5378999-95db-498a-ae14-f67b148bf6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:49.175715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be56d0-3125-44fd-a9d6-3294675df1c9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:49.189866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:49.190997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:49.191276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:49.191551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] PATH_INFO: `/v3/users/e2a11dedb3634ac895e3daa3e3f92861` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:49.193884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a11dedb3634ac895e3daa3e3f92861)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:49.202940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:49.204792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:49.232763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 850/1690] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:49.244398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:49.246984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:49.246984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:49.294778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:49.303073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:49.340652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 851/1691] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:49.342323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:49.388118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299bb26-8d9d-45a5-a200-bda6b2b2a1c4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:49.393468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:49.394158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:49.394443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:49.394760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:49.395549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:49.395866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:49.397516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:49.461935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 852/1692] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:49.467554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:49.508263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073a09d-c9c7-464c-bacb-bda40570494a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:49.518924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:49.520608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:49.520969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:49.521568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:49.522441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:49.524807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:49.608980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a11dedb3634ac895e3daa3e3f92861'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.633375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 853/1693] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:49 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:49.650254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:49.669816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a11dedb3634ac895e3daa3e3f92861'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.772551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a11dedb3634ac895e3daa3e3f92861'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.783690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a11dedb3634ac895e3daa3e3f92861'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.785053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a11dedb3634ac895e3daa3e3f92861'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:49.808116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 841/1694] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:48 2022] DELETE /identity/v3/users/e2a11dedb3634ac895e3daa3e3f92861 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:49.815021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb8f6f1-62cd-43a7-9372-a9cf77b84a1e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:50.050626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:50.051858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.057760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.058119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.065456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:50.071575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:50.158493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:50.164437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:50.237761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c49c9-84c1-40c0-a1d2-31716c8da2b9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:50.257599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 854/1695] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:49 2022] POST /identity/v3/users => generated 287 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:50.257865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:50.258684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:50.259869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.262097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.262425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] PATH_INFO: `/v3/projects/9c875d2a5043420e88f676fbc8cc25a5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.266090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c875d2a5043420e88f676fbc8cc25a5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:50.267059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:50.278483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:50.299653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c2974-7629-4caf-9a80-b944815f2a4b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:50.311594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:50.313644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.314346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.314763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.319335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:50.322812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:50.377262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 855/1696] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:50.378798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:50.391875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe65b06-4fb5-41d7-a12a-0b393d41a8df admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:50.398070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:50.399393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.399769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.400195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.401026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:50.403151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:50.453435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 856/1697] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:50.458143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:50.468604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c875d2a5043420e88f676fbc8cc25a5', 'request_id': 'req-858b6412-e684-4938-814e-b92efd7295aa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:50.484030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d807eff-a53a-4d7e-88c3-401aaa165a6b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:50.555681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c875d2a5043420e88f676fbc8cc25a5', 'request_id': 'req-858b6412-e684-4938-814e-b92efd7295aa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:50.580133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c875d2a5043420e88f676fbc8cc25a5', 'request_id': 'req-858b6412-e684-4938-814e-b92efd7295aa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:50.581685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:50.618080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] The token cache is being invalidate because project 9c875d2a5043420e88f676fbc8cc25a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:50.618855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:50.619462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-858b6412-e684-4938-814e-b92efd7295aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:50.621099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:50.623039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.624073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.624535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] PATH_INFO: `/v3/projects/480ed545bf9e447c831d7e2e9dd517b8/users/db5334b78c9c4cd39e0471493c87c99f/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.631011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 842/1698] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:50 2022] DELETE /identity/v3/projects/9c875d2a5043420e88f676fbc8cc25a5 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:50.637876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fdd6b-0a51-45b0-8ba2-d3ea9b245910 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.652818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fdd6b-0a51-45b0-8ba2-d3ea9b245910 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.652818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fdd6b-0a51-45b0-8ba2-d3ea9b245910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.653237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=480ed545bf9e447c831d7e2e9dd517b8, user_id=db5334b78c9c4cd39e0471493c87c99f, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:50.653237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcfa93-34a2-44b3-89c1-968288eb56fb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:50.699498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4c6fdd6b-0a51-45b0-8ba2-d3ea9b245910 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:50.699498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4c6fdd6b-0a51-45b0-8ba2-d3ea9b245910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:50.734353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 843/1699] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:50.737462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.738071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.738739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.761451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 857/1700] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:50 2022] PUT /identity/v3/projects/480ed545bf9e447c831d7e2e9dd517b8/users/db5334b78c9c4cd39e0471493c87c99f/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:50.773523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:50.773523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:50.773786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:50.833580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:50.835671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:50.874181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] MFA Rules not processed for user `db5334b78c9c4cd39e0471493c87c99f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:50.875768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:50.953525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:51.056354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 844/1701] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:51.057427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.079205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75473c9-ed6d-42b9-984d-1e1744b67335 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.079408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 858/1702] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:51.086122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.086919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.087208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.087488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.088341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:51.088657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.090850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.110132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 845/1703] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:51.114482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.132001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef5209-931c-4222-9cbd-39cc88226cd6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.138945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.139687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.139975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.140316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.141115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.143242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.250640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 859/1704] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:51 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:51.254991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.271675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ff1b7-ec2b-45da-90e8-8a2b38a68737 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.409956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.410725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.411005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.411354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.419420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.419420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.540070 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 846/1705] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:51 2022] POST /identity/v3/users => generated 287 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:51.544304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.559407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bb268-5aab-42b9-8fe9-fc59477fdcc2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.566038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.569817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.569817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.569817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.569817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.570768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.592536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 860/1706] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:51.592743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.612436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3da5a-4f21-482f-ab73-b3044be5b103 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.631336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.640503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.640916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.641262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.644284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.644284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.664173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.664624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 847/1707] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:51.684073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0181c6-cccc-44a0-b569-eed084fce65c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:51.690626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:51.693183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.693509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.694023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] PATH_INFO: `/v3/projects/f12fcf1830b04460828411eee3514d85/users/03d7559f4b114a148027183227e27d24/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.736127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12fcf1830b04460828411eee3514d85, user_id=03d7559f4b114a148027183227e27d24, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:51.739317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6838b-2995-410c-96b5-e46d9c8f6120 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:51.742772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:51.751175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:51.856357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 861/1708] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:51 2022] PUT /identity/v3/projects/f12fcf1830b04460828411eee3514d85/users/03d7559f4b114a148027183227e27d24/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:51.867485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:51.867956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:51.868357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:51.925519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:51.974136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] MFA Rules not processed for user `03d7559f4b114a148027183227e27d24`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:51.976470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:51.977410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047983a2-a600-4e42-89bb-5aa26aa58b5f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:52.045530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:52.151417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:52.159245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:52.159568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:52.159833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:52.161626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 862/1709] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:52.306774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:52.310156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:52.323343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 848/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:52.396811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:52.414416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a38d0-7b51-4800-b01c-ccd74e09cc11 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:52.430968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:52.432437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:52.432437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:52.432437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:52.445819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:52.447568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1528ba78-8d72-43e5-84f0-6321da614fd0 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:52.476661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 863/1711] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:52.483994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:52.486033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:52.695059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95fb0-bc6d-4453-b94e-1780e7ce084b service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:52.695059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:52.695059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:52.695059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:52.696322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] PATH_INFO: `/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:52.696322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c1d13b06324d9f902d43a9bc3fb73a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:52.696322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:53.135854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:53.155041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c1d13b06324d9f902d43a9bc3fb73a', 'request_id': 'req-a6e37adf-b9ef-41e3-a1c3-51b9af253238'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:53.252117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c1d13b06324d9f902d43a9bc3fb73a', 'request_id': 'req-a6e37adf-b9ef-41e3-a1c3-51b9af253238'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:53.269380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c1d13b06324d9f902d43a9bc3fb73a', 'request_id': 'req-a6e37adf-b9ef-41e3-a1c3-51b9af253238'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:53.288213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] The token cache is being invalidate because project e1c1d13b06324d9f902d43a9bc3fb73a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:53.288213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:53.289215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6e37adf-b9ef-41e3-a1c3-51b9af253238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:53.294061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 849/1712] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:52 2022] DELETE /identity/v3/projects/e1c1d13b06324d9f902d43a9bc3fb73a => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:53.299402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.303688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.303688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.336000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:53.337406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:53.475345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 864/1713] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:53.479420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.479649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.479890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.503250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:53.504512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:53.514604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 850/1714] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:53.518241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:53.530576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8402a-7c18-4192-a5cb-befdde8246de None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:53.536674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:53.536674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.536674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.537301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.537612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:53.538119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:53.539598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:53.557800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 865/1715] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:53.566424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:53.572915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b0159-b0c4-467e-bb19-01f752d1eb3a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:53.578103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:53.578797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.579077 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.579347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.580031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:53.582080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:53.659572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:53.663953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:53.677626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e7e8ab-c439-44f3-9e08-2c8212b2c3c2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:53.682244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:53.683184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.683463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.683735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] PATH_INFO: `/v3/users/db5334b78c9c4cd39e0471493c87c99f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.684927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5334b78c9c4cd39e0471493c87c99f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:53.686875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:53.727818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 851/1716] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:53 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:53.730411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:53.808263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05686b-b732-4c88-83d0-f23422c59da9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:53.813503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:53.814273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.814745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.815125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.815938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:53.818546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:53.966350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 852/1717] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:53 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:53.970304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:53.980786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b46e6-3c75-4bba-8206-b34dcd11ab88 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:53.986014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:53.991860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:53.991860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:53.991860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:53.991860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:53.991860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.000195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 853/1718] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:54.003910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5334b78c9c4cd39e0471493c87c99f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:54.004321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.014397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8664-d202-4cb7-bd65-cd4b63476d81 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.017688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5334b78c9c4cd39e0471493c87c99f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:54.020436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.026664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.026664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.026664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.026664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.026664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.037995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 854/1719] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:54.042765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.053758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bfbb2-f8f3-40d0-98b8-53834aea16c7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.060725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.060725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.060725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.060725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] PATH_INFO: `/v3/projects/a696728e5d1d45a9b3a0659ffd51e0b1/users/8d028a92d96c45d5a1f2336498e50be8/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.075344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a696728e5d1d45a9b3a0659ffd51e0b1, user_id=8d028a92d96c45d5a1f2336498e50be8, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.077996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509afa5-b7b0-43e3-8406-1a1eeb809ed4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.126281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5334b78c9c4cd39e0471493c87c99f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:54.134086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5334b78c9c4cd39e0471493c87c99f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:54.135286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7d1f2d18-4468-4ccf-8cd1-1f76457b41b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5334b78c9c4cd39e0471493c87c99f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:54.141142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 866/1720] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:53 2022] DELETE /identity/v3/users/db5334b78c9c4cd39e0471493c87c99f => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:54.145918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.146273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.146661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.164788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:54.166170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:54.172534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:54.175547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 855/1721] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:54 2022] PUT /identity/v3/projects/a696728e5d1d45a9b3a0659ffd51e0b1/users/8d028a92d96c45d5a1f2336498e50be8/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:54.189929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.190225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.190503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.218923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] MFA Rules not processed for user `8d028a92d96c45d5a1f2336498e50be8`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:54.220135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:54.232059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 867/1722] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:54.243017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.244822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:54.249377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:54.298010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 856/1723] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:54.303208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.304579 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:54.308386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cebc8-f4dd-4031-a154-c28b36171a9e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.314045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.314743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.315028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.315419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.318887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f274d5-12e9-48da-8ce6-f1846f096271 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.324173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.329196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.329196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.329196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.329196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.329509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.329592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.329668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.342616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 868/1724] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:54.424924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 857/1725] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:54 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:54.429702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.439977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a829c5-4c6a-4aee-9251-076602d66b32 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.515427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.516105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.516385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.516659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.517540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.524468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.609599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 869/1726] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:54 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:54.610902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.622437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e6e92-a4b1-4b02-ba40-0ae0bd240eb4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.627234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.627990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.628323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.628601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.630497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.631725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.646555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 858/1727] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:54.649146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.659871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b6239-8dcf-4b1a-8920-08881abb58c3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.665973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.667268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.667268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.667828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] PATH_INFO: `/v3/projects/ad602209e0ef492e8df4cb0dec772df3/users/156618faad094f1c80dd0a8357276987/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.686138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad602209e0ef492e8df4cb0dec772df3, user_id=156618faad094f1c80dd0a8357276987, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.688955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.795586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 870/1728] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:54 2022] PUT /identity/v3/projects/ad602209e0ef492e8df4cb0dec772df3/users/156618faad094f1c80dd0a8357276987/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:54.799603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.810428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12d273-809b-4174-81a9-433319f222ea admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.899184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.900276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.900982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.901745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.903586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.906509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82a19-9401-47b1-b638-8fcfec6d5646 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.938093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 859/1729] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:54.938684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:54.959320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddf10b-af04-428e-bbfe-6fd5d1c38f84 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:54.969712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:54.969712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:54.969712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:54.969712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] PATH_INFO: `/v3/projects/ad602209e0ef492e8df4cb0dec772df3/users/156618faad094f1c80dd0a8357276987/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:54.971714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad602209e0ef492e8df4cb0dec772df3, user_id=156618faad094f1c80dd0a8357276987, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:54.976466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:54.992438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 871/1730] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:54 2022] PUT /identity/v3/projects/ad602209e0ef492e8df4cb0dec772df3/users/156618faad094f1c80dd0a8357276987/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:55.012134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.013931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.013931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.027112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.029111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.064889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] MFA Rules not processed for user `156618faad094f1c80dd0a8357276987`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:55.067334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.132875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73d861-f139-4577-9b8d-b24d3240aa1b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.138844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.139885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.140432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.140723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] PATH_INFO: `/v3/projects/480ed545bf9e447c831d7e2e9dd517b8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.141981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=480ed545bf9e447c831d7e2e9dd517b8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.144041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:55.231368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 860/1731] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:55.243451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.245146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.278924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480ed545bf9e447c831d7e2e9dd517b8', 'request_id': 'req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:55.287187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480ed545bf9e447c831d7e2e9dd517b8', 'request_id': 'req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:55.327065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37e458-328e-4ea4-b87b-b36fba5e1245 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.392946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.393823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.398826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.398826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.399029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:55.399029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480ed545bf9e447c831d7e2e9dd517b8', 'request_id': 'req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:55.411043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] The token cache is being invalidate because project 480ed545bf9e447c831d7e2e9dd517b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:55.411043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:55.411870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a6bf48bf-72ab-4f3e-8d1b-e4617ff4ab64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:55.414592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 872/1732] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:55 2022] DELETE /identity/v3/projects/480ed545bf9e447c831d7e2e9dd517b8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:55.418432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.419374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.419769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.440059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:55.441300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.477964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.479509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd83516-a111-443b-9c7a-8e61022d22a4 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:55.491868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 861/1733] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:55.532844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 873/1734] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:55.534636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.534886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.535440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.581382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:55.583307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.601657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:55.607717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 862/1735] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:55.609155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.624885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72b6cc-4099-4346-aa6e-8d852c85f288 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.631746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.632421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.632785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.633067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.633881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:55.634101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.639855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:55.657670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 874/1736] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:43:55.661090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.678135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b5c131-eeb7-4386-93e8-3a44af0e2b36 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.683596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.683596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:55.684800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.685476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.685756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.686029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.686921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.689389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:55.698654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6047c38-65b4-4a60-a4f6-0000aaa1b770 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.708553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.713068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.713068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.713068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] PATH_INFO: `/v3/users/8d028a92d96c45d5a1f2336498e50be8` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.713068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d028a92d96c45d5a1f2336498e50be8)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.713815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:55.783551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 863/1737] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:55 2022] POST /identity/v3/projects => generated 403 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:55.790242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:55.869879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4496dc-8057-46e7-a229-a8ec85ce6127 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:55.874734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:55.880036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:55.880036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:55.880036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:55.880036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:55.880036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.006389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 864/1738] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:55 2022] POST /identity/v3/users => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:56.011471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.016330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d028a92d96c45d5a1f2336498e50be8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:56.030256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2321d-94f8-487e-97e1-d1bff0a2818e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.031050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d028a92d96c45d5a1f2336498e50be8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:56.045007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:56.049885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.049885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.049885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.049885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:56.051739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.067076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 865/1739] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:56.071474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.090088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3da425-9d22-4121-be4c-c81c2955b89e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.095254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:56.095956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.096244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.096525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.098587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:56.101062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.119253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 866/1740] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:56.124853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.148675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d028a92d96c45d5a1f2336498e50be8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:56.159000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df228c1-b3d5-43ce-afc0-09027bfd2d8c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.162823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d028a92d96c45d5a1f2336498e50be8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:56.164834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-94f6e896-2661-4775-9b4d-0bc73d5b6897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d028a92d96c45d5a1f2336498e50be8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:56.176769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:56.182088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 875/1741] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:55 2022] DELETE /identity/v3/users/8d028a92d96c45d5a1f2336498e50be8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:56.182384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.182384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.182384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] PATH_INFO: `/v3/projects/6182136be3714f5eaf9faef1b0dea5ff/users/100e4f39c5854b029cb4484e4abdbb20/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.185130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.185481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.185956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.201574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6182136be3714f5eaf9faef1b0dea5ff, user_id=100e4f39c5854b029cb4484e4abdbb20, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:56.205019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2df39-174e-4c3a-945c-18c8c06b7fd9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.211552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:56.213078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:56.278362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 876/1742] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:56.288660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.290085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:56.335313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 867/1743] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:56 2022] PUT /identity/v3/projects/6182136be3714f5eaf9faef1b0dea5ff/users/100e4f39c5854b029cb4484e4abdbb20/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:56.347495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.348880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.348880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.350395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93368a65-4cc5-44a6-8568-80bc410be1b2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.381437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] MFA Rules not processed for user `100e4f39c5854b029cb4484e4abdbb20`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:56.383252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:56.440687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:56.442274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.442419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.444154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.558275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:56.560295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.589723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 868/1744] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:56.595021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.598329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:56.618833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 877/1745] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:56.638730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3b4b-a554-4b3f-b06a-f27e6084904b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.650552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:56.653798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:56.654201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:56.654598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:56.655827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:56.663018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:56.900902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 869/1746] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:56 2022] POST /identity/v3/projects => generated 403 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:56.901853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:56.916862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c843117-aae1-4995-b347-5b206f0989c0 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:56.978620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:57.009194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.010194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.010571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.011066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.012662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.016577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.110558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 878/1747] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:43:56 2022] POST /identity/v3/users => generated 308 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:43:57.116021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.126418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f361f-21de-4cf4-babe-54264774d652 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.131985 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.132729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.133093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.133430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.134614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.135994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.149026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 870/1748] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:43:57.153109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.164598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb65611-cf54-455a-9752-1e3d2ba64c96 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.169470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.171597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.171597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.171597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] PATH_INFO: `/v3/projects/c98cc66299b14556a4cf140f48040e45/users/efeff35e4e3342ccb172898cf35eaaf2/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.186375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98cc66299b14556a4cf140f48040e45, user_id=efeff35e4e3342ccb172898cf35eaaf2, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.189125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.208494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.210158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.213710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:57.226095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04db31-f4cf-4b13-89eb-def3972b63cf admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.232570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.233801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.234096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.234375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] PATH_INFO: `/v3/projects/a696728e5d1d45a9b3a0659ffd51e0b1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.235677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a696728e5d1d45a9b3a0659ffd51e0b1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.237629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.268603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 879/1749] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:57 2022] PUT /identity/v3/projects/c98cc66299b14556a4cf140f48040e45/users/efeff35e4e3342ccb172898cf35eaaf2/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:57.273650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.336177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37076d7f-c85b-443c-a467-9eb08d91bc60 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.340856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.341544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.341829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.342107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.342870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.344473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.355059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 880/1750] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:43:57.372733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.383820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e986e8-d917-4da3-9cd3-854d89d060d7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.388471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.389371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.389640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.389916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] PATH_INFO: `/v3/projects/c98cc66299b14556a4cf140f48040e45/users/efeff35e4e3342ccb172898cf35eaaf2/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.391945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98cc66299b14556a4cf140f48040e45, user_id=efeff35e4e3342ccb172898cf35eaaf2, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.396230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a606cd4-0e7c-4eff-b39c-86ae800276cf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.407219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 881/1751] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:43:57 2022] PUT /identity/v3/projects/c98cc66299b14556a4cf140f48040e45/users/efeff35e4e3342ccb172898cf35eaaf2/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:57.418244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a696728e5d1d45a9b3a0659ffd51e0b1', 'request_id': 'req-28af3edb-5273-4dc4-aef4-335d86d2f711'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.420354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.420659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.420950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.449478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] MFA Rules not processed for user `efeff35e4e3342ccb172898cf35eaaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:57.455876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.525696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a696728e5d1d45a9b3a0659ffd51e0b1', 'request_id': 'req-28af3edb-5273-4dc4-aef4-335d86d2f711'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.532257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 882/1752] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:57 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:43:57.533868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a696728e5d1d45a9b3a0659ffd51e0b1', 'request_id': 'req-28af3edb-5273-4dc4-aef4-335d86d2f711'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.541785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.541785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.555170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] The token cache is being invalidate because project a696728e5d1d45a9b3a0659ffd51e0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:57.555351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.556183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.557672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 871/1753] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:57 2022] DELETE /identity/v3/projects/a696728e5d1d45a9b3a0659ffd51e0b1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:43:57.558480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.561597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.600023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f05d-11a7-4c7d-818d-7197d7dd9302 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.603557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.604211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.610500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.610500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.610500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.620059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.621510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.633899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 883/1754] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:57 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:57.634425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af3edb-5273-4dc4-aef4-335d86d2f711 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.634771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:57.636100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:57.639924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.640964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.641478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.641764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] PATH_INFO: `/v3/users/03d7559f4b114a148027183227e27d24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.643015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d7559f4b114a148027183227e27d24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.645781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.652242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80c4fb-4dd9-4272-a75c-1ca4f8bf81e1 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:57.656721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:57.657857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:57.658273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:57.658652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] PATH_INFO: `/v3/users/156618faad094f1c80dd0a8357276987` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:57.660011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=156618faad094f1c80dd0a8357276987)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:57.663749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:57.871236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d7559f4b114a148027183227e27d24'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.887437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d7559f4b114a148027183227e27d24'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:57.993277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '156618faad094f1c80dd0a8357276987'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.010541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '156618faad094f1c80dd0a8357276987'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.320602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d7559f4b114a148027183227e27d24'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.329359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d7559f4b114a148027183227e27d24'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.329721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d7559f4b114a148027183227e27d24'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.342286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 872/1755] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:57 2022] DELETE /identity/v3/users/03d7559f4b114a148027183227e27d24 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:58.343227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:58.442126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:58.442126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23770260-8164-439c-8ecf-a1b706ce47b0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:58.447779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:58.448691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:58.448975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:58.449250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] PATH_INFO: `/v3/users/100e4f39c5854b029cb4484e4abdbb20` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:58.450534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=100e4f39c5854b029cb4484e4abdbb20)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:58.452508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:58.851777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '156618faad094f1c80dd0a8357276987'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.859585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '156618faad094f1c80dd0a8357276987'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.860254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-96d1f531-7728-452f-98f3-1df46ff850f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '156618faad094f1c80dd0a8357276987'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:58.866870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 884/1756] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:57 2022] DELETE /identity/v3/users/156618faad094f1c80dd0a8357276987 => generated 0 bytes in 1232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:58.868322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:58.868884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:58.869158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:58.886734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:58.887964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:58.952237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 885/1757] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:58.962672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:58.964477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.003134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100e4f39c5854b029cb4484e4abdbb20'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.016188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100e4f39c5854b029cb4484e4abdbb20'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.025755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef9533-b04a-4910-a720-ed3a031d39e1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.030881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.031549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.031876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.032152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.042365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.043909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab9a5e-5ec3-4cec-8e14-55a951e9689b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.053977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 886/1758] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:43:59.068973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100e4f39c5854b029cb4484e4abdbb20'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.079173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100e4f39c5854b029cb4484e4abdbb20'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.080010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100e4f39c5854b029cb4484e4abdbb20'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.093373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 873/1759] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:43:58 2022] DELETE /identity/v3/users/100e4f39c5854b029cb4484e4abdbb20 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:59.096623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.096922 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.097200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.124609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:59.126722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.203578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 887/1760] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:59.214179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.215787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.327733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cbe9-1ccf-404c-b26a-42efa1bc65ee admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.333329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.334552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.334552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.334847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.347128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.350197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.369779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 874/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:43:59.385669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.387260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.401961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f32de-8ccc-4fcf-b1ff-1364e69c8fd4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.407268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.408153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.408437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.408825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] PATH_INFO: `/v3/projects/ad602209e0ef492e8df4cb0dec772df3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.410649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad602209e0ef492e8df4cb0dec772df3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.412672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.538134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad602209e0ef492e8df4cb0dec772df3', 'request_id': 'req-43360205-64e9-4a0f-b520-8cbaa32ee469'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.545965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad602209e0ef492e8df4cb0dec772df3', 'request_id': 'req-43360205-64e9-4a0f-b520-8cbaa32ee469'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.606015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad602209e0ef492e8df4cb0dec772df3', 'request_id': 'req-43360205-64e9-4a0f-b520-8cbaa32ee469'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.619960 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.621632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.622331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] The token cache is being invalidate because project ad602209e0ef492e8df4cb0dec772df3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:43:59.622668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.623351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43360205-64e9-4a0f-b520-8cbaa32ee469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.625682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 888/1762] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:59 2022] DELETE /identity/v3/projects/ad602209e0ef492e8df4cb0dec772df3 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:43:59.632017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4d777-f2d5-4094-a737-d8f9a6370fdc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.632317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4d777-f2d5-4094-a737-d8f9a6370fdc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.632603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4d777-f2d5-4094-a737-d8f9a6370fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.637081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f524d-d2b7-4ea5-8d33-cd6c68df4135 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.665153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:43:59.673080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e8a4d777-f2d5-4094-a737-d8f9a6370fdc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:59.674402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e8a4d777-f2d5-4094-a737-d8f9a6370fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.730056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.731228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.731512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.731793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] PATH_INFO: `/v3/projects/f12fcf1830b04460828411eee3514d85` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.733094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12fcf1830b04460828411eee3514d85)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.735281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.763726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 889/1763] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:59.768007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.768928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.769353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.790601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:43:59.791837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.800579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 890/1764] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:43:59.805566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.814779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90336f4-e4e8-4cb0-9e27-321d3255f3d5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.819639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.820280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.820561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.820835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.821630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:43:59.821933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.823588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.835942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 891/1765] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:43:59.840200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.851251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24480a5-1a43-44c4-aa61-dd7500f4e639 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:43:59.855885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:43:59.856554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:43:59.856834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:43:59.857107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:43:59.857784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:43:59.859699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:43:59.880002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12fcf1830b04460828411eee3514d85', 'request_id': 'req-4043c696-e476-4e17-a190-78bc3d7a6a2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.974450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 892/1766] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:43:59 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:43:59.975486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:43:59.977165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:43:59.997002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12fcf1830b04460828411eee3514d85', 'request_id': 'req-4043c696-e476-4e17-a190-78bc3d7a6a2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:43:59.998391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:00.007319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12fcf1830b04460828411eee3514d85', 'request_id': 'req-4043c696-e476-4e17-a190-78bc3d7a6a2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.020770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] The token cache is being invalidate because project f12fcf1830b04460828411eee3514d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:00.025489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.026976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.030141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 875/1767] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:59 2022] DELETE /identity/v3/projects/f12fcf1830b04460828411eee3514d85 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:00.031495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.034311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.065125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c37e3-b2ff-4095-adec-59b9f5cc7c8b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.069401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.070569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.070858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.071133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] PATH_INFO: `/v3/projects/6182136be3714f5eaf9faef1b0dea5ff` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.072412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6182136be3714f5eaf9faef1b0dea5ff)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.074312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.112667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043c696-e476-4e17-a190-78bc3d7a6a2d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.118265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.119399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.119696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.119979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.120687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.122896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df3688-bb97-401d-8882-92d5ca24cbde admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.231286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 876/1768] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:00.232599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160e58-205c-4c48-9915-25567dc16433 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.233060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160e58-205c-4c48-9915-25567dc16433 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.233349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01160e58-205c-4c48-9915-25567dc16433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.248973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6182136be3714f5eaf9faef1b0dea5ff', 'request_id': 'req-02ebdec3-850c-4e50-99c8-e274162f3131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.256831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-01160e58-205c-4c48-9915-25567dc16433 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:00.257399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6182136be3714f5eaf9faef1b0dea5ff', 'request_id': 'req-02ebdec3-850c-4e50-99c8-e274162f3131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.258240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-01160e58-205c-4c48-9915-25567dc16433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.319838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 877/1769] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:00.326156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160e58-205c-4c48-9915-25567dc16433 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.330112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01160e58-205c-4c48-9915-25567dc16433 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.336064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.336743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.337024 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.337296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.338025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.339805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66eb87-e29c-4a4a-88e2-316637175c13 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.350567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 878/1770] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:00.353145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.353145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.353145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.371250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6182136be3714f5eaf9faef1b0dea5ff', 'request_id': 'req-02ebdec3-850c-4e50-99c8-e274162f3131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.372969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:00.374188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.382775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 879/1771] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:00.384259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] The token cache is being invalidate because project 6182136be3714f5eaf9faef1b0dea5ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:00.384259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.387227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.387806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.387806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 893/1772] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:43:59 2022] DELETE /identity/v3/projects/6182136be3714f5eaf9faef1b0dea5ff => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:00.389656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.391673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.393670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c9d0f-b5a5-4350-b729-fb452c3aff65 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.399398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.400145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.400511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.400889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.401648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.403313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.404792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebdec3-850c-4e50-99c8-e274162f3131 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.409616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.411214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.411493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.411779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] PATH_INFO: `/v3/users/efeff35e4e3342ccb172898cf35eaaf2` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.412959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efeff35e4e3342ccb172898cf35eaaf2)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.414278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 880/1773] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:00.414946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.415136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.417736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.433016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d6c4e-77d2-4b34-af02-7cd8d930394b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.440870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.440870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.440870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.440870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.441317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:00.441317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.443088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.455475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 881/1774] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:00.458495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.459044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.473466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba330-c043-477e-a551-e6ef5b60a5d2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.478146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.480799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.480799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.480799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] PATH_INFO: `/v3/projects/f3456d9ef2254484b94fa9232e0920a6/users/786d5533844b4ffdafa2e1da3fb994e5/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.499805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3456d9ef2254484b94fa9232e0920a6, user_id=786d5533844b4ffdafa2e1da3fb994e5, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.502823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.613392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 882/1775] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:00 2022] PUT /identity/v3/projects/f3456d9ef2254484b94fa9232e0920a6/users/786d5533844b4ffdafa2e1da3fb994e5/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:00.615061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.662888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efeff35e4e3342ccb172898cf35eaaf2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.684651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efeff35e4e3342ccb172898cf35eaaf2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.688692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e91118-afe8-4b4b-9e34-dc89d0aa3912 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.694644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.695313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.695597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.695997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.696725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.698693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f812a3f-00d8-49e8-aded-d0e2ba994192 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:00.854970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efeff35e4e3342ccb172898cf35eaaf2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.865084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 883/1776] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/projects => generated 369 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:00.866748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.867133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.867560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.876904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efeff35e4e3342ccb172898cf35eaaf2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.877504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efeff35e4e3342ccb172898cf35eaaf2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:00.885811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 894/1777] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:00 2022] DELETE /identity/v3/users/efeff35e4e3342ccb172898cf35eaaf2 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:00.887342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:00.889347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.919773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] MFA Rules not processed for user `786d5533844b4ffdafa2e1da3fb994e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:00.921426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:00.989547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c6734c-fad1-4c78-8ca2-79458ab945d4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:00.994892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:00.995581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:00.995887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:00.996164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:00.996885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:00.999172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.022874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 884/1778] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:01.027295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.028593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.043021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9489c-82f9-4468-8701-c99d678c4961 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.046937 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.047548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.047825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.048100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.048782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.051557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.113699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 895/1779] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:00 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:01.115002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.143216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1930db-02ec-4ac9-930d-907df2f21b0d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.148941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.149656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.149988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.151095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.151851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.155425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.162270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 885/1780] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:01.165183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.168652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 896/1781] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:01.172722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.185402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48275dd3-3fa0-4217-beaf-b96edaac1d13 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.241135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be849e-1258-4d11-9fa7-9f8106eb3d53 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.246728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.247415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.247730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.248011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.248715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.251697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.265593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.266274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.266606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.266989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.267662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.269475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.283755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 897/1782] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:01.286061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.298750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc7939-0226-49fc-97f3-8c1589f7f236 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.302228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.303151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.303429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.303700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] PATH_INFO: `/v3/projects/f568451d5adc42c0b82b55b73ad33431/users/c57674b892f94e559cf207604fcd546c/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.319942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f568451d5adc42c0b82b55b73ad33431, user_id=c57674b892f94e559cf207604fcd546c, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.322616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.379023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:01.385069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 886/1783] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/users => generated 300 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:01.389092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.389092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.401377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd08b96-bbc9-434c-85ac-ec018fa5a56d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.405978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.406926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.407207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.407482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] PATH_INFO: `/v3/projects/c98cc66299b14556a4cf140f48040e45` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.408642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c98cc66299b14556a4cf140f48040e45)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.411407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.434592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 898/1784] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:01 2022] PUT /identity/v3/projects/f568451d5adc42c0b82b55b73ad33431/users/c57674b892f94e559cf207604fcd546c/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:01.440713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.505442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a742c5-49ff-414f-bd4d-78a9fab699e2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.510151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.510875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.511145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.511420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.512146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.513773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9a404-ff78-4e60-b3e9-3f0ef239b3b9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.528392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 899/1785] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:01.528392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.528392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.528392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.528780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c98cc66299b14556a4cf140f48040e45', 'request_id': 'req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:01.556688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] MFA Rules not processed for user `c57674b892f94e559cf207604fcd546c`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:01.557834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.617338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c98cc66299b14556a4cf140f48040e45', 'request_id': 'req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:01.622163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 900/1786] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:01.622998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.625148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c98cc66299b14556a4cf140f48040e45', 'request_id': 'req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:01.636611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] The token cache is being invalidate because project c98cc66299b14556a4cf140f48040e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:01.636978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:01.637506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:01.639142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 887/1787] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:01 2022] DELETE /identity/v3/projects/c98cc66299b14556a4cf140f48040e45 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:01.646307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.647079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.694310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a701a-17ed-46eb-b515-6a8ebe05c5a6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.699481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.701454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.701454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.701843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] PATH_INFO: `/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642/users/ab521b2e86084be68f820a38c60be1f9/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.720366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:01.723218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf102c6c-9ac7-4dcf-8745-607dc3682ed3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.728102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.728810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.729224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.729493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.730559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.732437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.732563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbabfc4e44f4fa6aa6f4785b5a0f642, user_id=ab521b2e86084be68f820a38c60be1f9, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.738390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32118b-36b4-430c-bb21-bed88921d5b8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.807701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 901/1788] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:01 2022] PUT /identity/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642/users/ab521b2e86084be68f820a38c60be1f9/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:01.808421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.808831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.809204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.828838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:01.830059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.840782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 888/1789] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:01.841957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.845447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:01.901736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 902/1790] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:01.903299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:01.932956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e836a1-abf1-45ee-aac6-0911d18c704b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.944428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359ceb0-112e-4c9a-a16f-2815ccc2842a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:01.949890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:01.950802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.951190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.951587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.952623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:01.954361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ad21f-01b8-4496-8ee9-06bf48d0f4ee admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:01.968594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 889/1791] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:01.968594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:01.968594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:01.968594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:01.991803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:01.993031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:02.013232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 890/1792] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:02.014325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.015363 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.016092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.016375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.016653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.017357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.020817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.026831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e12f8-482e-40d4-a97a-2c6490e7f021 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.031941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.032842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.033123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.033401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] PATH_INFO: `/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642/users/ab521b2e86084be68f820a38c60be1f9/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.036539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbabfc4e44f4fa6aa6f4785b5a0f642, user_id=ab521b2e86084be68f820a38c60be1f9, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.039312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.054587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 891/1793] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:02 2022] PUT /identity/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642/users/ab521b2e86084be68f820a38c60be1f9/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:02.055426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.126850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d616b0-bdf8-4061-9891-bcab77398b64 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.132018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.133017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.133304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.134327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.134327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:02.134789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.136583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb07ad5-f5d4-43e6-823a-2d49dbb59271 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.153534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 892/1794] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:02.153534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.153534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.153534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.173801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 903/1795] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:01 2022] POST /identity/v3/users => generated 290 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:02.176282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.188991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] MFA Rules not processed for user `ab521b2e86084be68f820a38c60be1f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:02.188991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:02.191744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c665a-625f-4472-ad3c-564490f99ff6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.204878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.204878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.204878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.204878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.205262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.205262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.262356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 893/1796] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:02.263044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:02.263324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.276207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dcae5c-5c4a-4499-bc38-72f397365d95 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.280887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.281560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.281839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.282137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.282907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.284500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.294981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 894/1797] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:02.295935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.298087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:02.309717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 904/1798] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:02 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:02.309717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.330142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:02.373712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe9dd8-2ffe-4537-9d0b-94229ec2c8cb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.389141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c4c5-bba0-48d3-be97-5ca9a9f4aefe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.394328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.398692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.398692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.398692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] PATH_INFO: `/v3/projects/ee52278704fe4871af7ea0be49493329/users/b22635468dc14df59d6e02d54c1bceb5/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.412202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee52278704fe4871af7ea0be49493329, user_id=b22635468dc14df59d6e02d54c1bceb5, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.415683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba848a4-55db-4d37-99fb-1fe3ba1bc420 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.435154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.435825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.436106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.436385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.502788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.505651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.522639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 895/1799] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:02.523573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 905/1800] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:02 2022] PUT /identity/v3/projects/ee52278704fe4871af7ea0be49493329/users/b22635468dc14df59d6e02d54c1bceb5/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:02.524646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca09ae-e07e-49f1-8235-2ca8aa988d8b None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.525545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.526087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca09ae-e07e-49f1-8235-2ca8aa988d8b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.526321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca09ae-e07e-49f1-8235-2ca8aa988d8b None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.528351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 906/1801] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:44:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:44:02.531355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca09ae-e07e-49f1-8235-2ca8aa988d8b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.555978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69bc78-6c20-468b-a1f9-842d6e0094dc service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.637605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.638559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.638888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.639011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca09ae-e07e-49f1-8235-2ca8aa988d8b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.639348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.641097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.642550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.648494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.649132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.649429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.649882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.650811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.656110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b999eaa-6903-45b6-9253-4a78e79a0662 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.669953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 907/1802] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:02.669953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.669953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.669953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.691779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:02.693526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:02.774563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 896/1803] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:02 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:02.775862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.779457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 908/1804] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:02.780595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.788860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb325-fe8c-4f20-bd25-bacc3a5eadc8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.795524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.795811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb079c-5c54-48f7-bd9a-ad9386609a25 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.797025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.797482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.797920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] PATH_INFO: `/v3/projects/ee52278704fe4871af7ea0be49493329/users/b22635468dc14df59d6e02d54c1bceb5/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.801290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee52278704fe4871af7ea0be49493329, user_id=b22635468dc14df59d6e02d54c1bceb5, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.801621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.802415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.803054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.803054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.803740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.804489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.805594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.817039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 897/1805] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:02 2022] PUT /identity/v3/projects/ee52278704fe4871af7ea0be49493329/users/b22635468dc14df59d6e02d54c1bceb5/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:02.817244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 909/1806] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:02.820824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.826803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:02.829114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:02.833019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a78fe0-1280-4861-a08c-fcbdfaf9dd11 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.946605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a166810-7bda-4705-82f8-9224c0cc7667 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:02.956562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:02.957397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.961620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.961620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:02.968454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:02.970069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fae6f1-d149-4931-8d68-ca234f6c34ce service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:02.982343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 898/1807] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:02.983420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:02.983714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:02.983996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.018104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] MFA Rules not processed for user `b22635468dc14df59d6e02d54c1bceb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:03.020309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:03.040125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.040980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.041366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.041746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.042711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.044720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.052652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:03.063241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 910/1808] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:44:03.064440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.076976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe045c-3689-47a6-853a-f71071fd8f3b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.082794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.083963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.084547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.084966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] PATH_INFO: `/v3/projects/9bfe25238021448aafe6f166fddc4207/users/bc9740d194fd400e90dfc94fc1253949/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.103705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bfe25238021448aafe6f166fddc4207, user_id=bc9740d194fd400e90dfc94fc1253949, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.113752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.116506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 899/1809] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:03.131777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.133091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:03.203283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad10a14-27a6-4ea8-9438-99681a5f47cb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.209865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.216750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.216750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.216750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.222600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.224648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc749d0-932d-48f2-87a6-28ba46c6acdc service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.235509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 900/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:03.310362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 911/1811] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:03 2022] PUT /identity/v3/projects/9bfe25238021448aafe6f166fddc4207/users/bc9740d194fd400e90dfc94fc1253949/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:03.322168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.322454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.322955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.352303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] MFA Rules not processed for user `bc9740d194fd400e90dfc94fc1253949`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:03.353524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:03.362670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:03.418073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.465713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 901/1812] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:03.470474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.519786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef028b-7db5-47f3-8305-5a4e38d07220 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.523493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.524146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.524433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.524709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.526964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:03.539759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a5443-264e-40bb-b9a4-0f5979e3b76d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.545795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.546619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.546992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.547336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.548365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.551596 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.599572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.601157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.615962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 912/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:03.650841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 902/1814] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:03 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:03.653148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.665352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8655e8-be4b-46ad-ac1c-be0a55df032e service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.730666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.731345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.731633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.731913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.732612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.734597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.835857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 913/1815] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:03 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:03.839812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.850285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54687fc0-8b4a-448b-85d1-11da4a6a9f77 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.854881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.855585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.855864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.856134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.856862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.858478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.870137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 903/1816] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:03.874428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:03.885005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02cefa-2cd9-4bfb-8d51-7a97c4dd9bdf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:03.889680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:03.890584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:03.890864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:03.891137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] PATH_INFO: `/v3/projects/e1838630c5b044ee91d43e7e2f10c441/users/41a47ad4b0044383a49b26186a00d29e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:03.905693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1838630c5b044ee91d43e7e2f10c441, user_id=41a47ad4b0044383a49b26186a00d29e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:03.908386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:03.919314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:04.040962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 914/1817] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:03 2022] PUT /identity/v3/projects/e1838630c5b044ee91d43e7e2f10c441/users/41a47ad4b0044383a49b26186a00d29e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:04.044947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.055956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd8f6c-d42b-48b0-878a-8d113695dafd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:04.126253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:04.126955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.127235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.127507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.128239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:04.129894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9fd64-3b37-44d2-9619-4a6508529cfd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:04.144469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 904/1818] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:04.145777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.158325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ebd55b-883a-4742-9a11-f5a9814cdaa0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:04.164026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:04.164916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.165200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.165476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] PATH_INFO: `/v3/projects/e1838630c5b044ee91d43e7e2f10c441/users/41a47ad4b0044383a49b26186a00d29e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.167512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1838630c5b044ee91d43e7e2f10c441, user_id=41a47ad4b0044383a49b26186a00d29e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:04.170159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:04.181602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 915/1819] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:04 2022] PUT /identity/v3/projects/e1838630c5b044ee91d43e7e2f10c441/users/41a47ad4b0044383a49b26186a00d29e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:04.192343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.192955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.193239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.220377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] MFA Rules not processed for user `41a47ad4b0044383a49b26186a00d29e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:04.221649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:04.302767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 905/1820] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:04.352088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.354915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.494776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766657d6-b4ac-4063-b41e-7c5cb4e82a75 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:04.499321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad734-0c7b-4599-b2e3-9937ae41469f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:04.502484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:04.503342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.503875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.504498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.516260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:04.517414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.517919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.518390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] PATH_INFO: `/v3/users/786d5533844b4ffdafa2e1da3fb994e5` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.520508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:04.521214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] RBAC: Authorizing `identity:delete_user(user_id=786d5533844b4ffdafa2e1da3fb994e5)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:04.522604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:04.528209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:04.528345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:04.567048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 906/1821] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:04.764980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786d5533844b4ffdafa2e1da3fb994e5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:04.780002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786d5533844b4ffdafa2e1da3fb994e5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:04.793969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.806830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b077-2555-43af-b113-13ddd5574361 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:04.808323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:04.808980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.809289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.809566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.851732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786d5533844b4ffdafa2e1da3fb994e5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:04.864353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786d5533844b4ffdafa2e1da3fb994e5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:04.865338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09ae026b-5eb1-474e-a473-ace0baa2621d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786d5533844b4ffdafa2e1da3fb994e5'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:04.874743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 916/1822] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:04 2022] DELETE /identity/v3/users/786d5533844b4ffdafa2e1da3fb994e5 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:04.880379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:04.880763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:04.881109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:04.891047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:04.892552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:04.900113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:04.901449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:04.902957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 907/1823] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:04.974656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 917/1824] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:04.981889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:04.983301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:05.041217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0caf4-dddf-4fc4-b19b-9e1f3f0c8468 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:05.108551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:05.109206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:05.109489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:05.109758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:05.128657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:05.130739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fa5bd-c993-4bd1-a831-e0e3fb3795f8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:05.147499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 908/1825] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:05.235458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=78,keystone:DELETE=17,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:05.437841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:05.636790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:05.638063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:05.652457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12ad36-839a-4542-8782-7371347748d2 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:05.658845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:05.660936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:05.661220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:05.662015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] PATH_INFO: `/v3/users/bc9740d194fd400e90dfc94fc1253949` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:05.663044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9740d194fd400e90dfc94fc1253949)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:05.665147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:05.958574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9740d194fd400e90dfc94fc1253949'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:05.970960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9740d194fd400e90dfc94fc1253949'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.058026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9740d194fd400e90dfc94fc1253949'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.071737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9740d194fd400e90dfc94fc1253949'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.073473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9740d194fd400e90dfc94fc1253949'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.087484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 918/1826] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:05 2022] DELETE /identity/v3/users/bc9740d194fd400e90dfc94fc1253949 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:06.092083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:06.092569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:06.092874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:06.131929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:06.137039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:06.257614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 909/1827] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:06.266985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:06.268462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:06.334855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:06.336438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:06.336592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09e77-00bb-44dc-b494-bbafd17c399d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:06.341106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:06.341962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:06.342691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:06.343115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:06.358544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:06.358544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:06.370669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 919/1828] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:06.384314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff8d59-e543-4720-a158-d38d3bf0c000 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:06.390247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:06.391204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:06.391491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:06.391769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] PATH_INFO: `/v3/projects/f3456d9ef2254484b94fa9232e0920a6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:06.393025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3456d9ef2254484b94fa9232e0920a6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:06.397447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:06.542637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3456d9ef2254484b94fa9232e0920a6', 'request_id': 'req-f8305815-c8d0-42c1-8ba5-c84f99b57332'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.629673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3456d9ef2254484b94fa9232e0920a6', 'request_id': 'req-f8305815-c8d0-42c1-8ba5-c84f99b57332'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.638379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3456d9ef2254484b94fa9232e0920a6', 'request_id': 'req-f8305815-c8d0-42c1-8ba5-c84f99b57332'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.659257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] The token cache is being invalidate because project f3456d9ef2254484b94fa9232e0920a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:06.659567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.660270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.662020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 910/1829] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:06 2022] DELETE /identity/v3/projects/f3456d9ef2254484b94fa9232e0920a6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:06.666411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:06.677487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ba5ad-1dd4-4af5-a0db-d22c524382ed service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:06.737306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:06.738974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:06.739290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:06.739527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] PATH_INFO: `/v3/users/ab521b2e86084be68f820a38c60be1f9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:06.740754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab521b2e86084be68f820a38c60be1f9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:06.742723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:06.886629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:06.888141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:06.902426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8305815-c8d0-42c1-8ba5-c84f99b57332 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:06.917175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:06.918109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:06.918393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:06.918709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] PATH_INFO: `/v3/projects/9bfe25238021448aafe6f166fddc4207` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:06.919916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bfe25238021448aafe6f166fddc4207)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:06.921837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:06.981590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab521b2e86084be68f820a38c60be1f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:06.995969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab521b2e86084be68f820a38c60be1f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.073871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bfe25238021448aafe6f166fddc4207', 'request_id': 'req-3901cfb1-77be-4d41-8a5e-fabef1f98748'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.193108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab521b2e86084be68f820a38c60be1f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.201144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab521b2e86084be68f820a38c60be1f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.201819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab521b2e86084be68f820a38c60be1f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.208371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 920/1830] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:06 2022] DELETE /identity/v3/users/ab521b2e86084be68f820a38c60be1f9 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:07.286633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bfe25238021448aafe6f166fddc4207', 'request_id': 'req-3901cfb1-77be-4d41-8a5e-fabef1f98748'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.297466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bfe25238021448aafe6f166fddc4207', 'request_id': 'req-3901cfb1-77be-4d41-8a5e-fabef1f98748'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.317652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] The token cache is being invalidate because project 9bfe25238021448aafe6f166fddc4207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:07.317947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.318748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.324531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 911/1831] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:06 2022] DELETE /identity/v3/projects/9bfe25238021448aafe6f166fddc4207 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:07.327524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:07.329170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:07.411359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1073e-cbe3-497a-a05f-e3d653235ad8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:07.417251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:07.418375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:07.422070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:07.422070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] PATH_INFO: `/v3/users/41a47ad4b0044383a49b26186a00d29e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:07.422070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a47ad4b0044383a49b26186a00d29e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:07.422535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:07.497437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:07.757784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a47ad4b0044383a49b26186a00d29e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.770882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a47ad4b0044383a49b26186a00d29e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.812804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:07.814293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:07.882762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901cfb1-77be-4d41-8a5e-fabef1f98748 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:07.888009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:07.888869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:07.889148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:07.889448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] PATH_INFO: `/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:07.890697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbabfc4e44f4fa6aa6f4785b5a0f642)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:07.891401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a47ad4b0044383a49b26186a00d29e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.892648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:07.901681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a47ad4b0044383a49b26186a00d29e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.902423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23e1ec7f-d744-4afd-8ac3-8c1f8104749e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a47ad4b0044383a49b26186a00d29e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:07.908604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 921/1832] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:07 2022] DELETE /identity/v3/users/41a47ad4b0044383a49b26186a00d29e => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:08.019801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbabfc4e44f4fa6aa6f4785b5a0f642', 'request_id': 'req-b8c89961-1d94-4a1c-93e2-4c046da16d25'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.131755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbabfc4e44f4fa6aa6f4785b5a0f642', 'request_id': 'req-b8c89961-1d94-4a1c-93e2-4c046da16d25'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.134812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:08.142657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbabfc4e44f4fa6aa6f4785b5a0f642', 'request_id': 'req-b8c89961-1d94-4a1c-93e2-4c046da16d25'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.157677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] The token cache is being invalidate because project 8cbabfc4e44f4fa6aa6f4785b5a0f642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:08.157677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.157677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b8c89961-1d94-4a1c-93e2-4c046da16d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.165744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 912/1833] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:07 2022] DELETE /identity/v3/projects/8cbabfc4e44f4fa6aa6f4785b5a0f642 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:08.172312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.172940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.173299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.198988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:08.200255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.265855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 922/1834] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:08.273641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.274044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.274588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.293371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:08.294713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.303739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 913/1835] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:08.308469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.327579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ebe7a-315d-4fc9-9d71-1b71003c34e1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.333586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.334379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.334813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.335073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.335873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:08.336306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.338085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.351085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 923/1836] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:08.355598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.368786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b85380-0797-4dd4-be9c-e332c0018c4f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.373284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.375238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.377127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.378635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.379375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.380031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.381303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.383509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.396049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c509ad-cebb-4299-8c0d-d92a2cd8818a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.403449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.404598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.404806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.405142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] PATH_INFO: `/v3/projects/e1838630c5b044ee91d43e7e2f10c441` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.406533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1838630c5b044ee91d43e7e2f10c441)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.410258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.509337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 914/1837] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:08.513414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.575561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1838630c5b044ee91d43e7e2f10c441', 'request_id': 'req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.581165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1838630c5b044ee91d43e7e2f10c441', 'request_id': 'req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.605438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933e321-73ba-4ef0-b3cf-5adcd9bc2693 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.662251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1838630c5b044ee91d43e7e2f10c441', 'request_id': 'req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.670099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.671043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.671487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.678803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.678803 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.679006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] The token cache is being invalidate because project e1838630c5b044ee91d43e7e2f10c441 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:08.679006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.679154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.679242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-89fb6d2e-5dee-4755-b492-aca3f7aa91ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:08.679242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 924/1838] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:08 2022] DELETE /identity/v3/projects/e1838630c5b044ee91d43e7e2f10c441 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:08.682470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f520d-55b7-4bda-a367-6a45715617e5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.682827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f520d-55b7-4bda-a367-6a45715617e5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.683100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f520d-55b7-4bda-a367-6a45715617e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.701941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ad7f520d-55b7-4bda-a367-6a45715617e5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:08.703404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ad7f520d-55b7-4bda-a367-6a45715617e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.711782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 925/1839] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:08.719626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.719915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.720218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.753291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:08.753291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.759005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 926/1840] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:08.759874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.770191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c940e-53f0-4cd1-9a46-8b09ed4060e0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.775101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.778812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.778812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.778812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.778812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:08.778812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.779216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.792548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 927/1841] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:08.799224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.806687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 915/1842] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:08.813261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.815887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:08.817338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14eb0d-7d3f-4085-9b73-286c8a9348d5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.829755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.830442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.830724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7379c0-2216-49cd-a41f-73531d2a7f88 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.830833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.831572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.832877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.835735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.838284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.839026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.839308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.839580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.840469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.842332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.854558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 916/1843] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:08.858174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.869568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c79fb-025d-431f-948a-22222d4b3e56 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.874605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.875284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.875565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.875872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.876627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.878363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.888698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 917/1844] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:44:08.892735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.903790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f77226-ca7f-4388-ac63-57bf28dbbd39 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.908598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.912765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 928/1845] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:08.912936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.912936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.912936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] PATH_INFO: `/v3/projects/f1af73f8ecf2431fbb426229675f58c5/users/a29cee658a9142dabcd4b3cf84111cd4/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.913890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:08.931162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1af73f8ecf2431fbb426229675f58c5, user_id=a29cee658a9142dabcd4b3cf84111cd4, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.933943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac962-0c34-4d85-b41a-e131afae14e6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:08.977681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc88e45-a426-479c-851d-cfb6ced1e233 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:08.982602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:08.983351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:08.983629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:08.983905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:08.984589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:08.986589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.011548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 918/1846] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:08 2022] PUT /identity/v3/projects/f1af73f8ecf2431fbb426229675f58c5/users/a29cee658a9142dabcd4b3cf84111cd4/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:09.025480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.025768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.026043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.052801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] MFA Rules not processed for user `a29cee658a9142dabcd4b3cf84111cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:09.054012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.071641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 929/1847] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:08 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:09.072572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.083105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35421dea-e299-4692-83cb-9fcf562cc3df admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.124177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 919/1848] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:09.135428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.136834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.150804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.151909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.152427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.152710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.153456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.155122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.167419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 930/1849] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:09.171986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.183849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429edadd-208c-4d1f-97f6-b3bb052e00bf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.193661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.194425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.194779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.195032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.195779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.197495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.204381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4d0beb-5bb5-4b64-b040-14735d627a7f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.208823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.209484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.209776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.209918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 931/1850] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:09.210129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.214131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.219955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.221444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.223956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b3f7-573a-4327-8acb-478344148561 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.228592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.229469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.229752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.230028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] PATH_INFO: `/v3/projects/89af27bb6f8a4251b9aa8a92d1570aa7/users/ff933ba224b64072b991e11e7aa3ad5c/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.230833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 920/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:09.245809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89af27bb6f8a4251b9aa8a92d1570aa7, user_id=ff933ba224b64072b991e11e7aa3ad5c, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.248561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa123053-9ef5-459a-8ec3-fe0203032442 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.296839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.297302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:09.298413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.313305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7f72f-4519-47da-8d4d-002984a0f6e7 service glance] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.320366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.321309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.321601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.321878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] PATH_INFO: `/v3/users/c57674b892f94e559cf207604fcd546c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.323160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57674b892f94e559cf207604fcd546c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.325485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.333141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 932/1852] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:09 2022] PUT /identity/v3/projects/89af27bb6f8a4251b9aa8a92d1570aa7/users/ff933ba224b64072b991e11e7aa3ad5c/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:09.346173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.346823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.347127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.388191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] MFA Rules not processed for user `ff933ba224b64072b991e11e7aa3ad5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:09.389399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.458448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 933/1853] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:09.469967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.471430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.538715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5b0aa-0323-42d7-aec6-cf3166104b22 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:09.543096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:09.543751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.544178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.545145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.554662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:09.556211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e2851-f7dc-40d2-8c5a-52b1296d8b3b service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:09.568658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 934/1854] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:09.588466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:09.595046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57674b892f94e559cf207604fcd546c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:09.609973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57674b892f94e559cf207604fcd546c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:09.805279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57674b892f94e559cf207604fcd546c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:09.813129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57674b892f94e559cf207604fcd546c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:09.813799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57674b892f94e559cf207604fcd546c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:09.819999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 921/1855] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:09 2022] DELETE /identity/v3/users/c57674b892f94e559cf207604fcd546c => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:09.825200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:09.825494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:09.825796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:09.850024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:09.852505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.921790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 935/1856] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:09.931342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:09.933212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:09.999604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaf4c0-fafa-4336-a81b-9679b89a2bfd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:10.003871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:10.004633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:10.004826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:10.005101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:10.017108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:10.019423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:10.045674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 922/1857] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:10.550260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:10.552454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:10.570944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d2b04-7670-48c2-a9ae-529149d38ffe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:10.579305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825907bb-bd62-486d-8054-239648792048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:10.580578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825907bb-bd62-486d-8054-239648792048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:10.580779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825907bb-bd62-486d-8054-239648792048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:10.581243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825907bb-bd62-486d-8054-239648792048 admin admin] PATH_INFO: `/v3/projects/f568451d5adc42c0b82b55b73ad33431` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:10.582551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825907bb-bd62-486d-8054-239648792048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f568451d5adc42c0b82b55b73ad33431)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:10.584500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825907bb-bd62-486d-8054-239648792048 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:10.617508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:10.746079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f568451d5adc42c0b82b55b73ad33431', 'request_id': 'req-825907bb-bd62-486d-8054-239648792048'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:10.753113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f568451d5adc42c0b82b55b73ad33431', 'request_id': 'req-825907bb-bd62-486d-8054-239648792048'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:10.885523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f568451d5adc42c0b82b55b73ad33431', 'request_id': 'req-825907bb-bd62-486d-8054-239648792048'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:10.897108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] The token cache is being invalidate because project f568451d5adc42c0b82b55b73ad33431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:10.897393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:10.898039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:10.899640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 936/1858] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:10 2022] DELETE /identity/v3/projects/f568451d5adc42c0b82b55b73ad33431 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:10.903781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:10.915717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f03b1-6670-4e65-b8d6-c6c60d6d9872 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:10.979474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:10.980377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:10.980657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:10.980931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] PATH_INFO: `/v3/users/b22635468dc14df59d6e02d54c1bceb5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:10.982635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b22635468dc14df59d6e02d54c1bceb5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:10.984464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:10.989875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:11.253841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b22635468dc14df59d6e02d54c1bceb5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:11.267187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b22635468dc14df59d6e02d54c1bceb5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:11.404268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b22635468dc14df59d6e02d54c1bceb5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:11.412600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b22635468dc14df59d6e02d54c1bceb5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:11.413238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-28de2d18-62df-4658-9ff8-707fdd26d530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b22635468dc14df59d6e02d54c1bceb5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:11.419367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 923/1859] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:10 2022] DELETE /identity/v3/users/b22635468dc14df59d6e02d54c1bceb5 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:11.634257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:11.726377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:11.881357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:11.883029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:11.899861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825907bb-bd62-486d-8054-239648792048 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:11.961757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:11.962654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:11.962944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:11.963313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] PATH_INFO: `/v3/projects/ee52278704fe4871af7ea0be49493329` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:11.964610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee52278704fe4871af7ea0be49493329)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:11.966547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.010862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:12.114673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee52278704fe4871af7ea0be49493329', 'request_id': 'req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:12.121803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee52278704fe4871af7ea0be49493329', 'request_id': 'req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:12.199746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee52278704fe4871af7ea0be49493329', 'request_id': 'req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:12.217386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] The token cache is being invalidate because project ee52278704fe4871af7ea0be49493329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:12.217671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:12.218393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b9311b86-1410-45e4-aaf2-ebc5e245e9b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:12.223921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 937/1860] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:11 2022] DELETE /identity/v3/projects/ee52278704fe4871af7ea0be49493329 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:12.224081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:12.225483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.225763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.226225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.245855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:12.247133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:12.313737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 924/1861] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:12.322701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a29afe-f232-4042-8746-76b6685faced None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.323327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a29afe-f232-4042-8746-76b6685faced None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.324603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a29afe-f232-4042-8746-76b6685faced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.346143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c6a29afe-f232-4042-8746-76b6685faced None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:12.347654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6a29afe-f232-4042-8746-76b6685faced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:12.356748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 938/1862] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:12.361597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.372516 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0b794-f0ff-4eda-9e8e-f210b1e834b2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.377413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.378755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.378903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.379182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.379913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:12.380223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.381907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.403522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 925/1863] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:12.406961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a29afe-f232-4042-8746-76b6685faced None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.423101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a29afe-f232-4042-8746-76b6685faced None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.428959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.429623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.432267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.432267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.432267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.432856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.585995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 939/1864] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:12 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:12.588871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.591210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:12.601273 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630586e8-8828-4e55-9c4f-b5e0eed76ab6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.660370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.660999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.661287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.661563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.662312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.664321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.788914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 926/1865] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:12 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:12.792934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.803907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9d17-c8c0-4063-b1c1-b8b87531652f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.808664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.809392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.809675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.809953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.810701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.812295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.824918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 940/1866] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:12.834634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.845115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b5ff-2a4e-49a9-8ecb-53135637511a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.850379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.851079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.851365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.851643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.852375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.853976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33ca30-bb41-47e7-9836-cf114e23d0c5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:12.864987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 927/1867] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:12.872822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:12.883988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cec2c-eeda-4e6b-8b14-47dcd4e390ad admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:12.888507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:12.891387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:12.891387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:12.891387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] PATH_INFO: `/v3/projects/6be1c98f0a4c4632b37d4ab7a580fc48/users/bf4363614082411e86ab91674ff35d84/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:12.904556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be1c98f0a4c4632b37d4ab7a580fc48, user_id=bf4363614082411e86ab91674ff35d84, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:12.907446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.000200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 941/1868] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:12 2022] PUT /identity/v3/projects/6be1c98f0a4c4632b37d4ab7a580fc48/users/bf4363614082411e86ab91674ff35d84/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:13.012338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.012707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.012805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.041259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] MFA Rules not processed for user `bf4363614082411e86ab91674ff35d84`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:13.042492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:13.109265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 928/1869] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:13.114752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.126341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:13.178669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ddb6a1-c555-40a9-ae8e-288e5370c89c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.183709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.184367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.184675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.185004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.185814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.187767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.299279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 942/1870] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:13 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:13.303908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.315901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c05d74-bf5f-4a45-9409-0fe05cb41c04 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.380248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.381032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.381315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.381585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.382286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.384274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.491978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 929/1871] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:13 2022] POST /identity/v3/users => generated 300 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:13.500819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.509654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a4d7c-c26f-4045-b02a-e0ae5d88f2d2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.515541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.515541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.515541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.515541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.516546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.519891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.535940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 943/1872] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:44:13.538131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.550952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62213-fce8-47ba-997b-4c63a655bc26 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.557507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.558473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.563951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.563951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] PATH_INFO: `/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580/users/e3a48d1ada964dee9ef881d7b1027e47/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.575608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4de9d7ed0c74b1bbd6357a2b4686580, user_id=e3a48d1ada964dee9ef881d7b1027e47, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.579378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.600149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:13.674683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 930/1873] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:13 2022] PUT /identity/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580/users/e3a48d1ada964dee9ef881d7b1027e47/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:13.681305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.693471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2e9bf-a02f-48b9-8071-7aab2ade8737 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.757131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.757806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.758137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.758412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.759178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.760862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565fa21c-59c1-4fc7-83a7-b5e6fd1a4a18 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.771513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 944/1874] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:13.775664 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.786537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ab1a-57eb-4a76-809f-7f2f7cbab2d3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.791075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.792283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.792556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.792967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] PATH_INFO: `/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580/users/e3a48d1ada964dee9ef881d7b1027e47/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.794996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4de9d7ed0c74b1bbd6357a2b4686580, user_id=e3a48d1ada964dee9ef881d7b1027e47, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:13.797734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:13.808170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 931/1875] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:13 2022] PUT /identity/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580/users/e3a48d1ada964dee9ef881d7b1027e47/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:13.819267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.819737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.819826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:13.849116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] MFA Rules not processed for user `e3a48d1ada964dee9ef881d7b1027e47`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:13.850356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:13.920170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 945/1876] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:13.931483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:13.933222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:13.948256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:13.992169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb52c5-a64b-4ea6-8943-fd4dab475ee3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:13.996113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:13.996758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:13.997039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:13.997314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:14.007573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:14.009047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cefebe7-968a-4d1c-9e4e-55819503c7bf service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:14.019492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 932/1877] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:14.039637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:14.047123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:14.109840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f544dc-efeb-47ee-9295-99032a75e890 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:14.115344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:14.116210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:14.116492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:14.116778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] PATH_INFO: `/v3/users/bf4363614082411e86ab91674ff35d84` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:14.118342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4363614082411e86ab91674ff35d84)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:14.120327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:14.181204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:14.500806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:14.773902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4363614082411e86ab91674ff35d84'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:14.794258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4363614082411e86ab91674ff35d84'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:14.936365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4363614082411e86ab91674ff35d84'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:14.950672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4363614082411e86ab91674ff35d84'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:14.951466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4363614082411e86ab91674ff35d84'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:14.963393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 946/1878] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:14 2022] DELETE /identity/v3/users/bf4363614082411e86ab91674ff35d84 => generated 0 bytes in 914 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:14.965528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:14.965814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:14.966087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:14.984963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:14.986194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:15.046829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 933/1879] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:15.056662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:15.058426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:15.120422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfca7b-37b6-4512-ac30-df6784abbe6a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:15.125439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:15.126631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:15.126973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:15.127249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:15.137502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:15.139033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:15.149774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 947/1880] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:15.462863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:15.587225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:15.732203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:15.734028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:15.769126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e9f25-3ce4-460e-904d-9c37ed2e5194 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:15.785170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:15.787245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:15.787537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:15.787816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] PATH_INFO: `/v3/projects/6be1c98f0a4c4632b37d4ab7a580fc48` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:15.789925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be1c98f0a4c4632b37d4ab7a580fc48)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:15.800716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:16.003953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be1c98f0a4c4632b37d4ab7a580fc48', 'request_id': 'req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.119864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be1c98f0a4c4632b37d4ab7a580fc48', 'request_id': 'req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.128598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be1c98f0a4c4632b37d4ab7a580fc48', 'request_id': 'req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.143510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] The token cache is being invalidate because project 6be1c98f0a4c4632b37d4ab7a580fc48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:16.144166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.145097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.147113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 934/1881] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:15 2022] DELETE /identity/v3/projects/6be1c98f0a4c4632b37d4ab7a580fc48 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:16.151391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:16.164133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa73c3-7e84-405d-b797-e8151c61efae service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:16.199706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:16.234777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:16.235521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:16.235804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:16.246011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] PATH_INFO: `/v3/users/e3a48d1ada964dee9ef881d7b1027e47` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:16.247077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a48d1ada964dee9ef881d7b1027e47)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:16.249231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:16.448280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a48d1ada964dee9ef881d7b1027e47'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.461641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a48d1ada964dee9ef881d7b1027e47'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.536811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:16.593665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a48d1ada964dee9ef881d7b1027e47'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.603804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a48d1ada964dee9ef881d7b1027e47'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.604278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-6e49f9d4-5605-4ce3-b1ae-da08ae43472f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a48d1ada964dee9ef881d7b1027e47'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:16.612522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 948/1882] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:16 2022] DELETE /identity/v3/users/e3a48d1ada964dee9ef881d7b1027e47 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:16.932811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:17.312343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:17.313949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:17.342160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee542ef0-b185-4d31-afe1-0bf5a48b3c54 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:17.470008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:17.474395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:17.474710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:17.479069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] PATH_INFO: `/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:17.480775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4de9d7ed0c74b1bbd6357a2b4686580)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:17.483534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:17.829595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4de9d7ed0c74b1bbd6357a2b4686580', 'request_id': 'req-cccec93d-9565-4159-bedf-ff9eeda768cb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:17.851090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:17.948101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4de9d7ed0c74b1bbd6357a2b4686580', 'request_id': 'req-cccec93d-9565-4159-bedf-ff9eeda768cb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:17.955058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4de9d7ed0c74b1bbd6357a2b4686580', 'request_id': 'req-cccec93d-9565-4159-bedf-ff9eeda768cb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:17.966473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] The token cache is being invalidate because project b4de9d7ed0c74b1bbd6357a2b4686580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:17.966819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:17.967476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cccec93d-9565-4159-bedf-ff9eeda768cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:17.969122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 935/1883] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:17 2022] DELETE /identity/v3/projects/b4de9d7ed0c74b1bbd6357a2b4686580 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:17.975102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:17.975634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:17.975755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:17.995724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:17.996953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:18.065369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 949/1884] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:18.073185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.073484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.073760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.094019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:18.095345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:18.106713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 936/1885] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:18.108802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.121616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136364d-15f4-4ffa-bd2c-fcce5af062f7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.128041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.128822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.129281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.129926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.131083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:18.132348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.134352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.151906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 950/1886] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:18.155232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.166989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0223bbb-339d-4b46-a5a4-8abba0857aa6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.175634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.176558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.176828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.177753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.178465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.180639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.323742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 937/1887] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:18 2022] POST /identity/v3/projects => generated 401 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:18.327535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.338785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89664ed4-3f62-43f8-837f-232bd34d6865 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.424756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.425510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.425849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.426133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.428086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.430655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.472549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.475580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:18.499364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb679ec8-f13c-4d97-91f8-f58d32248611 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.508205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.514974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.514974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.514974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] PATH_INFO: `/v3/users/ff933ba224b64072b991e11e7aa3ad5c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.514974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff933ba224b64072b991e11e7aa3ad5c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.526222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.533593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:18.542731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 951/1888] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:18 2022] POST /identity/v3/users => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:18.546555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.569519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8e5af-0644-4e42-a489-7f14bcd59286 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.575802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.577133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.577803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.578336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.579789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.582484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.596213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 952/1889] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:18.603483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.622413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f57137-344a-4fa8-a258-872e15ae5c91 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.629207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.630153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.630640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.631068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.632060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.633909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.656208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 953/1890] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:18.657313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:18.702408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80e2b9-1211-4b60-94bb-81361256b3d1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:18.726785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:18.727700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.727982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.728259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] PATH_INFO: `/v3/projects/5962fe80c57942e9a61f91a72676d6c1/users/686243a60a8542b39986c868ad83f195/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.762152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5962fe80c57942e9a61f91a72676d6c1, user_id=686243a60a8542b39986c868ad83f195, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:18.766549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574037-9bb3-4a7c-ac45-b80d136f3b89 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:18.807656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff933ba224b64072b991e11e7aa3ad5c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:18.829963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff933ba224b64072b991e11e7aa3ad5c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:18.879576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 954/1891] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:18 2022] PUT /identity/v3/projects/5962fe80c57942e9a61f91a72676d6c1/users/686243a60a8542b39986c868ad83f195/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:18.898843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:18.898843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:18.898843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:18.923558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] MFA Rules not processed for user `686243a60a8542b39986c868ad83f195`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:18.925559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:18.981718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff933ba224b64072b991e11e7aa3ad5c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:18.991266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff933ba224b64072b991e11e7aa3ad5c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:18.992543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6603974c-76b8-4d52-b09d-f06e54f413ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff933ba224b64072b991e11e7aa3ad5c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:18.999095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 938/1892] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:18 2022] DELETE /identity/v3/users/ff933ba224b64072b991e11e7aa3ad5c => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:19.003700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.004574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.005054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.014297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 955/1893] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:19.018751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.031093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:19.032539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:19.103761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72cf76-594a-48e0-b572-2c1a66f157f9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.118177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 939/1894] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:19.130226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.131677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:19.141603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=3,keystone:DELETE=15,keystone:SELECT=82 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:19.168828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.169646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.173310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.173310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.173310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.173310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.193126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:19.197928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79148142-ebc8-4f23-8d5c-543df0bb546b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.201506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.202178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.202463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.202771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.212318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.213746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.224295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 940/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:19.265766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 956/1896] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:19 2022] POST /identity/v3/projects => generated 401 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:19.269603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.271082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:19.287702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d0afa-329c-4010-b4d1-fcab33ab2e96 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.348649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.349334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.349601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.349880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.350613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.352552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.445718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 941/1897] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:19 2022] POST /identity/v3/users => generated 307 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:19.449963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.462086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b844f13-1702-49cf-a63f-743114b06041 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.467036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.467707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.467985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.468261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.468979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.470591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.484321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 957/1898] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:19.500237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.510798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4221c33-731c-48db-8309-1ac62d40c191 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.515680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.516661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.517046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.517462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] PATH_INFO: `/v3/projects/8f57b688c3c8422eb708472c4dce0efb/users/0666eb24294d4938a2ab07616407f99b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.543792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f57b688c3c8422eb708472c4dce0efb, user_id=0666eb24294d4938a2ab07616407f99b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.546782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.656947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 942/1899] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:19 2022] PUT /identity/v3/projects/8f57b688c3c8422eb708472c4dce0efb/users/0666eb24294d4938a2ab07616407f99b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:19.661698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.671630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a8a15-895b-438c-8860-1c86647203de admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.679169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:19.745645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.746369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.746694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.746994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.747728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.749346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.758680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.760877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:19.764439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 958/1900] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:19.765377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.777284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:19.781055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864013-045c-4465-9c92-6b27118ab39a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.794557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.799754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.799754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.799754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] PATH_INFO: `/v3/projects/8f57b688c3c8422eb708472c4dce0efb/users/0666eb24294d4938a2ab07616407f99b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.799754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f57b688c3c8422eb708472c4dce0efb, user_id=0666eb24294d4938a2ab07616407f99b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.800494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67f903-7575-4128-9a4e-e41950a47c89 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:19.803301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4227e4-2d38-4f99-84d5-cc031a2df795 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.804325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:19.805215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.805494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.806012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] PATH_INFO: `/v3/projects/89af27bb6f8a4251b9aa8a92d1570aa7` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.807231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89af27bb6f8a4251b9aa8a92d1570aa7)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:19.809140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:19.818700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 959/1901] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:19 2022] PUT /identity/v3/projects/8f57b688c3c8422eb708472c4dce0efb/users/0666eb24294d4938a2ab07616407f99b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:19.828928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:19.829425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:19.829879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:19.864114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] MFA Rules not processed for user `0666eb24294d4938a2ab07616407f99b`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:19.865428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:19.938814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 960/1902] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:19.952727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:19.954140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:20.026492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86602d14-df0a-4a26-b2a4-0e7e9b0605e5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.033646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89af27bb6f8a4251b9aa8a92d1570aa7', 'request_id': 'req-ffdd149b-d4a2-498a-8408-3288eca2c338'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:20.081144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.082108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.082276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.082690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.139158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89af27bb6f8a4251b9aa8a92d1570aa7', 'request_id': 'req-ffdd149b-d4a2-498a-8408-3288eca2c338'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:20.147921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89af27bb6f8a4251b9aa8a92d1570aa7', 'request_id': 'req-ffdd149b-d4a2-498a-8408-3288eca2c338'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:20.155314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.157020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81a50c-cebd-48f9-8813-02d09c49dc2d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.160110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] The token cache is being invalidate because project 89af27bb6f8a4251b9aa8a92d1570aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:20.160467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:20.161187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ffdd149b-d4a2-498a-8408-3288eca2c338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:20.164617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 943/1903] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:19 2022] DELETE /identity/v3/projects/89af27bb6f8a4251b9aa8a92d1570aa7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:20.168871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.169162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.169440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.171208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 961/1904] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:19 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:20.204855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:20.206279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:20.275997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 944/1905] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:20.284531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.284824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.285138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.305825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:20.307205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:20.317694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 962/1906] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:20.325505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.336360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969fec8-4b6b-4295-b051-411ffc43eef3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.341625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.342280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.342800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.343086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.343840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:20.344147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.345844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.358599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 945/1907] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:20.363545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.373494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755028ee-21a9-4685-9fd7-ee360b19d6cc None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.378873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.379542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.379827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.380117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.380806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.382783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.437510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.439260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:20.453149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd4038-f23f-4632-81a1-db4e3d36b643 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.464495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.465180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.474417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.474417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.474417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:20.530738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.532193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.532957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 963/1908] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:20 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:20.537026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.543675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 946/1909] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:20.615059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69462e86-5e26-4ec3-9179-802f78b753de admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.620571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.621242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.621599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.621874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.622586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.624495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.776988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 964/1910] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:20 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:20.781000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.792288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a37e1-3715-4c28-9c9b-135b344046a3 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.796761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.797443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.797721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.797997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.799438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.801183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.815175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 947/1911] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:20.816491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.827816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c4cd8-2dc7-4d16-ac69-91f627391ca2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.833950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.834725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.835013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.835282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.836006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.837602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7f172-c30e-4e32-a2ac-6388ce24c806 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:20.849025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 965/1912] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:20.859590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:20.879783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cfaaf-2a57-4a51-b094-5230758cb9a4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:20.884407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:20.885306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:20.885584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:20.885857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] PATH_INFO: `/v3/projects/24fda46b568945cf94b6e493f2e29654/users/31353b77f3404c22929d08c234c79b37/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:20.903832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24fda46b568945cf94b6e493f2e29654, user_id=31353b77f3404c22929d08c234c79b37, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:20.906492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:21.006608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 948/1913] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:20 2022] PUT /identity/v3/projects/24fda46b568945cf94b6e493f2e29654/users/31353b77f3404c22929d08c234c79b37/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:21.026194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:21.026626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:21.027449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:21.039936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:21.082362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] MFA Rules not processed for user `31353b77f3404c22929d08c234c79b37`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:21.083640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:21.151426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 966/1914] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:21.163356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:21.165045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:21.199555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:21.230243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f77f8-cda4-4477-be4f-03e52440bab0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:21.234772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:21.235447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:21.235729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:21.239292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:21.245840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:21.247345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af401f5c-3ce7-4ade-9dd7-fe07d432a8a5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:21.257415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 949/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:21.292794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:21.405527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846dc4a-e205-41fc-b0e5-db1ffb9f1b34 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:21.411025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:21.411923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:21.412282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:21.412815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] PATH_INFO: `/v3/users/31353b77f3404c22929d08c234c79b37` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:21.417600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31353b77f3404c22929d08c234c79b37)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:21.421471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:21.670559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31353b77f3404c22929d08c234c79b37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:21.696354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:21.702831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31353b77f3404c22929d08c234c79b37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:21.771987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31353b77f3404c22929d08c234c79b37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:21.786369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31353b77f3404c22929d08c234c79b37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:21.787456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31353b77f3404c22929d08c234c79b37'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:21.795495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 967/1916] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:21 2022] DELETE /identity/v3/users/31353b77f3404c22929d08c234c79b37 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:21.800052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:21.800568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:21.800644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:21.837320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:21.840202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:21.926575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 950/1917] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:21.941562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:21.944335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:22.009034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e9196-b605-4ec4-ad5d-3bba9caa1f72 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:22.012638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:22.013288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:22.013571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:22.013845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:22.024806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:22.026240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f84e4e-f306-4c53-a99b-7936db5dd23b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:22.036188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 968/1918] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:22.079838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:22.297510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:22.811758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:22.813267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:22.827914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06abe1c-067a-43e2-a5fa-6fa601bb2b62 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:22.833857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:22.834770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:22.835080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:22.835355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] PATH_INFO: `/v3/projects/24fda46b568945cf94b6e493f2e29654` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:22.836662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24fda46b568945cf94b6e493f2e29654)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:22.838965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:22.958791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:22.972003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24fda46b568945cf94b6e493f2e29654', 'request_id': 'req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:23.051029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24fda46b568945cf94b6e493f2e29654', 'request_id': 'req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:23.057965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24fda46b568945cf94b6e493f2e29654', 'request_id': 'req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:23.068600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] The token cache is being invalidate because project 24fda46b568945cf94b6e493f2e29654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:23.069408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:23.069566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6e5e043-4460-4d2b-98e4-a479f30fdb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:23.071780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 951/1919] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:22 2022] DELETE /identity/v3/projects/24fda46b568945cf94b6e493f2e29654 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:23.076609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.077101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.077192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.096599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:23.098410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:23.170992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 969/1920] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:23.175666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.176181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.176670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.227639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:23.229036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:23.239741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 952/1921] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:23.243934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.255679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9cb1d-f4bd-41fe-a2f8-fcf80e72d285 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.261558 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.262301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.262628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.262936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.263677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:23.263989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.265738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.294953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 970/1922] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:23.296129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.319558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3894d0-6fe4-4178-8d34-fd4d118de7cf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.333159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.333943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.334394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.334695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.335528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.339989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.473313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 953/1923] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:23 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:23.475683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.490029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fabfa1-b907-4420-8e00-b4c9dc44e582 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.529827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:23.559878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.560674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.561028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.561377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.562163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.564487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.680806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 971/1924] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:23 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:23.685141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.695727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145238b2-568b-4cce-83ea-d19d3e986116 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.700800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.701469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.701759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.702042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.702843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.704448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.727896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 954/1925] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:44:23.728724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.740240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646746-583f-41c6-aa9d-8218bee2f17e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.747519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.749041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.749494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.749780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.750546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.752535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41b22d-36d2-4a34-bc4e-7cb9df648ef7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.770637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 972/1926] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:23.774063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:23.787740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29400-6a24-4758-ba22-3de0dd7bc084 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:23.793109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:23.794182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.794556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.795143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] PATH_INFO: `/v3/projects/6d53cb44cc53495c87e2ede1b0ccc547/users/e8934466baba45ca9540f8667ced8b79/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.819575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d53cb44cc53495c87e2ede1b0ccc547, user_id=e8934466baba45ca9540f8667ced8b79, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:23.822645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:23.924388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 955/1927] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:23 2022] PUT /identity/v3/projects/6d53cb44cc53495c87e2ede1b0ccc547/users/e8934466baba45ca9540f8667ced8b79/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:23.926601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:23.926897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:23.927251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:23.972050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] MFA Rules not processed for user `e8934466baba45ca9540f8667ced8b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:23.973463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:24.062325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 973/1928] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:24.910959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:24.915296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:24.995523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2a448-5b96-45b4-8294-db4c0ac3cd1e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:24.999101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:24.999760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:25.000148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:25.000429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:25.010878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:25.012360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6cbcb6-2cf3-4920-99ab-6b78b45483f0 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:25.023307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 956/1929] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:34.758497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:34.760134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:34.836112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774d5c7-38d7-4da3-95c1-b0b68cd366ff None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:34.840990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:34.841854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:34.842138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:34.842413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] PATH_INFO: `/v3/users/686243a60a8542b39986c868ad83f195` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:34.843672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=686243a60a8542b39986c868ad83f195)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:34.845643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:35.019034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:35.069846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:35.073545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '686243a60a8542b39986c868ad83f195'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:35.088419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '686243a60a8542b39986c868ad83f195'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:35.181226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '686243a60a8542b39986c868ad83f195'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:35.192626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '686243a60a8542b39986c868ad83f195'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:35.192626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '686243a60a8542b39986c868ad83f195'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:35.200798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 974/1930] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:34 2022] DELETE /identity/v3/users/686243a60a8542b39986c868ad83f195 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:35.205496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:35.205794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:35.207468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:35.233106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:35.234903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:35.304559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 957/1931] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:35.314652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:35.316990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:35.385332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c092d43-2e79-448a-b6f1-1994986667bc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:35.389033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:35.389744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:35.389972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:35.390248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:35.401479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:35.403029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:35.414336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 975/1932] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:36.187755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:36.189340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:36.204124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5dee8-7181-4a22-81d0-7bcbdc2f111c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:36.210314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:36.211229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:36.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:36.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] PATH_INFO: `/v3/projects/5962fe80c57942e9a61f91a72676d6c1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:36.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5962fe80c57942e9a61f91a72676d6c1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:36.216828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:36.371687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5962fe80c57942e9a61f91a72676d6c1', 'request_id': 'req-2e6688b5-bd81-4a64-9f96-6c1e1534b266'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.444141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5962fe80c57942e9a61f91a72676d6c1', 'request_id': 'req-2e6688b5-bd81-4a64-9f96-6c1e1534b266'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.459214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5962fe80c57942e9a61f91a72676d6c1', 'request_id': 'req-2e6688b5-bd81-4a64-9f96-6c1e1534b266'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.472963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] The token cache is being invalidate because project 5962fe80c57942e9a61f91a72676d6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:36.473558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.474398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.476648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 958/1933] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:36 2022] DELETE /identity/v3/projects/5962fe80c57942e9a61f91a72676d6c1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:36.480519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:36.495966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa8c94-2999-4f0b-a752-3c6909e795a0 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:36.564302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:36.566052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:36.566625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:36.567014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] PATH_INFO: `/v3/users/0666eb24294d4938a2ab07616407f99b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:36.568355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0666eb24294d4938a2ab07616407f99b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:36.570313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:36.850409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0666eb24294d4938a2ab07616407f99b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.863821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0666eb24294d4938a2ab07616407f99b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.867339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:36.950560 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0666eb24294d4938a2ab07616407f99b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.973324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0666eb24294d4938a2ab07616407f99b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.974250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e82fc31c-cad9-4dbc-ab93-97ab8d907660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0666eb24294d4938a2ab07616407f99b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:36.981889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 976/1934] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:36 2022] DELETE /identity/v3/users/0666eb24294d4938a2ab07616407f99b => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:37.638774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:37.641258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:37.671472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6688b5-bd81-4a64-9f96-6c1e1534b266 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:37.804789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:37.806008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:37.806346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:37.806797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] PATH_INFO: `/v3/projects/8f57b688c3c8422eb708472c4dce0efb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:37.810845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f57b688c3c8422eb708472c4dce0efb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:37.813018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:37.972914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:38.015995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f57b688c3c8422eb708472c4dce0efb', 'request_id': 'req-5edfa8d0-becb-4757-88cf-084705d5f316'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:38.096013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f57b688c3c8422eb708472c4dce0efb', 'request_id': 'req-5edfa8d0-becb-4757-88cf-084705d5f316'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:38.107455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f57b688c3c8422eb708472c4dce0efb', 'request_id': 'req-5edfa8d0-becb-4757-88cf-084705d5f316'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:38.120204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] The token cache is being invalidate because project 8f57b688c3c8422eb708472c4dce0efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:38.120504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:38.121192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5edfa8d0-becb-4757-88cf-084705d5f316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:38.123064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 959/1935] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:37 2022] DELETE /identity/v3/projects/8f57b688c3c8422eb708472c4dce0efb => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:38.128968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.129800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.130728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.153738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:38.155144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:38.221208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 977/1936] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:38.231323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.231619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.231908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.250962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:38.252190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:38.260244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 960/1937] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:38.264751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.275635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e82ad-ba45-4a2d-93ee-968d632b7db1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.281085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.281808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.282089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.282361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.283148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:38.283451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.285813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.297551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 978/1938] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:38.302637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.312224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c7cf8-025c-435e-83b4-0327b22e9dce None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.317487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.318142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.318425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.318732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.319425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.321370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.431506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 961/1939] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:38.432676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.443712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beab34-f65e-4d38-8ec8-82f5bf0f4122 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.503865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.504643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.504958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.505238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.505982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.507973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.608998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 979/1940] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:38.613306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.625529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7801dd5-25ab-4bd3-b1b8-c7fb559d64f1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.634768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.634768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.634768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.634768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.635305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.635305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.649604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 962/1941] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:38.652183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.663200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303b136-01bd-4d94-a885-75fe4b7f239c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.672112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.672112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.672112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.672112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.672549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.673996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34301c60-9725-494a-bf0a-12b07eb411b0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.688290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 980/1942] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:38.689476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:38.707546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e41fa-98a5-4e75-a704-a02c354379db admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:38.712468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:38.713367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.713661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.713916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] PATH_INFO: `/v3/projects/86404f31d8a24031a73f6aed4db3d9e6/users/1070f8b2d2e2448fb324a0069546d418/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.730539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=86404f31d8a24031a73f6aed4db3d9e6, user_id=1070f8b2d2e2448fb324a0069546d418, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:38.733368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:38.843497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 963/1943] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:38 2022] PUT /identity/v3/projects/86404f31d8a24031a73f6aed4db3d9e6/users/1070f8b2d2e2448fb324a0069546d418/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:38.855904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:38.856338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:38.856826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:38.890037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] MFA Rules not processed for user `1070f8b2d2e2448fb324a0069546d418`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:38.892853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:38.947314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:38.971926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 981/1944] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:38.979619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.057379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d62b93-39b8-428f-8867-a8d4f8f53bec admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.062339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.063058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.063344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.065964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.065964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.067823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.211945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 964/1945] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:38 2022] POST /identity/v3/projects => generated 367 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:39.217927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.238310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5cb64-d7ed-47c8-8645-ed2decf8092c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.304317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.305006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.305293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.305572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.306272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.308430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.411901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 982/1946] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:39 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:39.414634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.428341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee47323-a1e3-4483-a0c0-2ba4abaee1f4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.438651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.439417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.439702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.440962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.441789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.443634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.456529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 965/1947] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:39.461367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.473739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156c815-4bba-4e15-8b06-9de29427e171 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.479801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.480752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.481036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.481314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] PATH_INFO: `/v3/projects/5dba4451411342ff96bed9793c6ae22b/users/fdeae95c27fa4b14a1512cb196565b3c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.499609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dba4451411342ff96bed9793c6ae22b, user_id=fdeae95c27fa4b14a1512cb196565b3c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.502321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.592931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 983/1948] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:39 2022] PUT /identity/v3/projects/5dba4451411342ff96bed9793c6ae22b/users/fdeae95c27fa4b14a1512cb196565b3c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:39.597280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.607884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c0d66-b705-4854-8c9a-82b2a2b26ef1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.635493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:39.679399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.680075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.680410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.680688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.681612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.683195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ddf33a-6317-447f-9372-9bb0db2a7540 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.695214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 966/1949] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:39.699582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.710899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d631f-4011-4fb6-b0b9-d740bca6e2e9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.715809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.716809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.717160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.717525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] PATH_INFO: `/v3/projects/5dba4451411342ff96bed9793c6ae22b/users/fdeae95c27fa4b14a1512cb196565b3c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.719621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dba4451411342ff96bed9793c6ae22b, user_id=fdeae95c27fa4b14a1512cb196565b3c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.722858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.738101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 984/1950] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:39 2022] PUT /identity/v3/projects/5dba4451411342ff96bed9793c6ae22b/users/fdeae95c27fa4b14a1512cb196565b3c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:39.748976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.749705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.750060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.790449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] MFA Rules not processed for user `fdeae95c27fa4b14a1512cb196565b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:39.790449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:39.868221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 967/1951] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:39.889330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:39.891394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:39.962633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56371b67-3805-4137-838a-addd4bf4af9f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:39.966692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:39.967458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:39.967630 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:39.967908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:39.978775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:39.980221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:39.991741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 985/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:40.124727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:40.126229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:40.201212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c845b7-1471-4793-bb0d-46f6802ae005 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:40.204004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:40.204896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:40.205177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:40.205453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] PATH_INFO: `/v3/users/56cc2c6c31dd4970b01fdf23fa6eda0d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:40.206697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=56cc2c6c31dd4970b01fdf23fa6eda0d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:40.208618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:40.256285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:40.264807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171955b2-6c66-47f4-bdcd-e4324fba01b2 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:40.269060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:40.269700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:40.270148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:40.270483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:40.312276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:40.342371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:40.344210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28b2f2-ab8d-453c-89b1-018593758deb service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:40.357618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 986/1953] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:40.443693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56cc2c6c31dd4970b01fdf23fa6eda0d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:40.446697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:40.458822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56cc2c6c31dd4970b01fdf23fa6eda0d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:40.552078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56cc2c6c31dd4970b01fdf23fa6eda0d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:40.559734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56cc2c6c31dd4970b01fdf23fa6eda0d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:40.560357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56cc2c6c31dd4970b01fdf23fa6eda0d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:40.566980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 968/1954] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:40 2022] DELETE /identity/v3/users/56cc2c6c31dd4970b01fdf23fa6eda0d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:40.571561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:40.571915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:40.572280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:40.594433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:40.595698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:40.662499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 987/1955] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:40.679706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:40.681770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:40.743250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ac19a-47c5-486e-9c16-34579182bb0a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:40.746979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:40.747637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:40.747917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:40.748190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:40.760425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:40.769777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:40.773694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 969/1956] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:41.245285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:41.248124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:41.265057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21376ee8-61ec-48fc-a41d-8abc66835b45 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:41.270817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:41.271686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:41.275185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:41.275185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] PATH_INFO: `/v3/projects/59f8d8f062af47dda1c3e63327c83245` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:41.275185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59f8d8f062af47dda1c3e63327c83245)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:41.275541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:41.420546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59f8d8f062af47dda1c3e63327c83245', 'request_id': 'req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.427997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59f8d8f062af47dda1c3e63327c83245', 'request_id': 'req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.522193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59f8d8f062af47dda1c3e63327c83245', 'request_id': 'req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.547720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] The token cache is being invalidate because project 59f8d8f062af47dda1c3e63327c83245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:41.547720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.547720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.547720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 988/1957] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:41 2022] DELETE /identity/v3/projects/59f8d8f062af47dda1c3e63327c83245 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:41.550283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:41.560786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc635-93d0-485f-be39-25ff9763cb6f service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:41.619881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:41.620768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:41.621093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:41.621412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] PATH_INFO: `/v3/users/3a29fb2f733f4971b003bd060d7cc492` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:41.622624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a29fb2f733f4971b003bd060d7cc492)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:41.624543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:41.852392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a29fb2f733f4971b003bd060d7cc492'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.864825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a29fb2f733f4971b003bd060d7cc492'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.868793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:41.957861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a29fb2f733f4971b003bd060d7cc492'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.965751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a29fb2f733f4971b003bd060d7cc492'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.966380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9b60f807-192c-4fe1-8d12-d29555f82997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a29fb2f733f4971b003bd060d7cc492'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:41.973211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 970/1958] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:41 2022] DELETE /identity/v3/users/3a29fb2f733f4971b003bd060d7cc492 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:42.428568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:42.430037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:42.443271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b6c03-fcf2-41c5-8b5a-de94a5b8db97 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:42.501348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:42.507458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:42.508354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:42.508635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:42.508913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] PATH_INFO: `/v3/projects/7d4f4ce1fe984e279335d66603bbb39c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:42.510106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4f4ce1fe984e279335d66603bbb39c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:42.512055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:42.620880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4f4ce1fe984e279335d66603bbb39c', 'request_id': 'req-0156bb97-44bb-44d8-b277-8fa37f61c4cd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:42.627294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4f4ce1fe984e279335d66603bbb39c', 'request_id': 'req-0156bb97-44bb-44d8-b277-8fa37f61c4cd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:42.688920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4f4ce1fe984e279335d66603bbb39c', 'request_id': 'req-0156bb97-44bb-44d8-b277-8fa37f61c4cd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:42.700915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] The token cache is being invalidate because project 7d4f4ce1fe984e279335d66603bbb39c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:42.701201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:42.701873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0156bb97-44bb-44d8-b277-8fa37f61c4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:42.703490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 989/1959] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:42 2022] DELETE /identity/v3/projects/7d4f4ce1fe984e279335d66603bbb39c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:42.709367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:42.709665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:42.709948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:42.729392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:42.730639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:42.793456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 971/1960] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:42.801796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:42.801961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:42.802232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:42.822968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:42.824192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:42.838087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 990/1961] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:42.842281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:42.854030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b5040-5035-4a52-965f-5928d7c8f670 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:42.858253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:42.859025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:42.859291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:42.866593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:42.866593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:42.866593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:42.866593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:42.875183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 972/1962] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:42.879910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:42.889667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b23761-b31f-440d-afeb-7964db12f02e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:42.895028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:42.895735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:42.896018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:42.896291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:42.896978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:42.899013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.061985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 991/1963] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:42 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:43.067298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.077792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c9a4e-6593-4238-b8fc-bb17f4aca474 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.136525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.137199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.137466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.137740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.138419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.140496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.233011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 973/1964] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:43 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:43.238076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.248979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9d3af1-3514-4032-966e-db0eabb84f8b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.254323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.255033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.255321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.255599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.256351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.257960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.268559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 992/1965] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:44:43.272747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.282773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aead7-c4b8-4e97-b69d-a1755f4d2bd4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.287602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.288279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.288564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.288854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.289591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.291244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95324c49-462d-43ae-81ed-41167a34338a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.302919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 974/1966] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:43.307249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.318422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de1eb-57a6-4945-9784-6a72fb894429 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.323459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.324463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.324745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.325021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] PATH_INFO: `/v3/projects/b201a29a1d934b6aa719fe6e2e0eb713/users/0bfcd00d9a2c41e38c9084aa65a718fd/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.341153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b201a29a1d934b6aa719fe6e2e0eb713, user_id=0bfcd00d9a2c41e38c9084aa65a718fd, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.343854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.438038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 993/1967] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:43 2022] PUT /identity/v3/projects/b201a29a1d934b6aa719fe6e2e0eb713/users/0bfcd00d9a2c41e38c9084aa65a718fd/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:43.449685 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.449980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.450262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.478676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] MFA Rules not processed for user `0bfcd00d9a2c41e38c9084aa65a718fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:43.479972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:43.523377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:43.546510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 975/1968] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:43.551411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.613962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cdcac1-5a9e-48f3-b3d3-8a13d76436ad admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.618908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.619572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.619853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.620129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.620822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.622757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:43.725176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 994/1969] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:43 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:43.729220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:43.739851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c3b7d-7872-429f-a099-7b55586ed29a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:43.804082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:43.805047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:43.805326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:43.805599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:43.806426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:43.808354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.018408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 976/1970] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:43 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:44.021149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.031423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f62d5-57b5-4320-9a64-9580a21fb2b5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.036358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.037027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.037307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.037574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.038296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.039925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.043242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:44.052252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 995/1971] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:44.056504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.067211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50cfb17-0284-49be-9afd-eb45ea552f1a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.071789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.072449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.072731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.073005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.073729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.075368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ab2c9-cedf-4b37-b06e-580457d99409 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.085491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 977/1972] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:44.091834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.102471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85f886-8515-48f5-a2d1-f76dbec14f59 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.107141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.108017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.108300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.108574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] PATH_INFO: `/v3/projects/b343f924d73b4d1cb8a633ecf8c0403c/users/90246be1fd2849bb941dd3166e2985a1/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.123902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b343f924d73b4d1cb8a633ecf8c0403c, user_id=90246be1fd2849bb941dd3166e2985a1, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.126473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.230841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 996/1973] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:44 2022] PUT /identity/v3/projects/b343f924d73b4d1cb8a633ecf8c0403c/users/90246be1fd2849bb941dd3166e2985a1/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:44.241953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.242239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.242545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.276762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] MFA Rules not processed for user `90246be1fd2849bb941dd3166e2985a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:44.278132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:44.359931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 978/1974] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:44.366008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.432297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a682-f285-4fce-83cd-7007bb597602 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.437438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.438704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.439263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.439263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.440007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.442052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.558703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 997/1975] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:44 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:44.562777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.573728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966b0b1-48a7-4d8c-a0d9-133efcd59a9c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.621507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:44.631373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.632100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.632559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.632838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.633583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.635583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.735097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 979/1976] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:44 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:44.741348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.757624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179fc7d-a50b-41de-8c70-4017a8359805 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.762341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.763027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.763303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.766846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.766846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.766846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.777407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 998/1977] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:44.781538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.792208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ee04e-b092-43c9-b8d7-5e5ed6535c7e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:44.796770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:44.797645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:44.797925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:44.798199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] PATH_INFO: `/v3/projects/7d23ed3f65434d3a952544b856cb6974/users/ae94aea892594ad08c5c7826ed673769/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:44.815106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d23ed3f65434d3a952544b856cb6974, user_id=ae94aea892594ad08c5c7826ed673769, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:44.818324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:44.924066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 980/1978] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:44 2022] PUT /identity/v3/projects/7d23ed3f65434d3a952544b856cb6974/users/ae94aea892594ad08c5c7826ed673769/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:44.928011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:44.937964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb6f4b-75f7-48d5-8ce8-c0bd6f1015cc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:45.004853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:45.010545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:45.010545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:45.010545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:45.010545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:45.010545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec0f3c-f655-4bd3-883f-53a387705771 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:45.025213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 999/1979] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:45.034116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:45.042638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd034a87-ba4e-499a-946c-aa53f8538600 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:45.052368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:45.053815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:45.054307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:45.054412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] PATH_INFO: `/v3/projects/7d23ed3f65434d3a952544b856cb6974/users/ae94aea892594ad08c5c7826ed673769/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:45.057327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d23ed3f65434d3a952544b856cb6974, user_id=ae94aea892594ad08c5c7826ed673769, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:45.060026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:45.078945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 981/1980] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:45 2022] PUT /identity/v3/projects/7d23ed3f65434d3a952544b856cb6974/users/ae94aea892594ad08c5c7826ed673769/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:45.083246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:45.083573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:45.083923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:45.117311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] MFA Rules not processed for user `ae94aea892594ad08c5c7826ed673769`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:45.118690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:45.210511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1000/1981] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:45.279065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:45.280792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:45.344037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b96bf-2286-449f-b871-4aec8c24232f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:45.347865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:45.348525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:45.348807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:45.349083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:45.358320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:45.362508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0607c0c-453b-498b-b3ce-5b49276777a5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:45.370494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 982/1982] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:45.828991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:45.830509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:45.897187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8a94a-fcc5-4e50-a0bf-24f510c3dbce None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:45.902232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:45.903175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:45.903455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:45.903752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] PATH_INFO: `/v3/users/a29cee658a9142dabcd4b3cf84111cd4` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:45.904942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29cee658a9142dabcd4b3cf84111cd4)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:45.907077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:46.169290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:46.189618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29cee658a9142dabcd4b3cf84111cd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:46.204233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29cee658a9142dabcd4b3cf84111cd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:46.304771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29cee658a9142dabcd4b3cf84111cd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:46.315305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29cee658a9142dabcd4b3cf84111cd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:46.316078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29cee658a9142dabcd4b3cf84111cd4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:46.325629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1001/1983] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:45 2022] DELETE /identity/v3/users/a29cee658a9142dabcd4b3cf84111cd4 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:46.329939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:46.330442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:46.330563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:46.350671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:46.351885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:46.419155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 983/1984] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:46.429277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:46.443005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:46.496679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6459b-fce6-4ce5-9661-cd9dc6212ce5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:46.501106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:46.501932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:46.502223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:46.502482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:46.513807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:46.515483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:46.533176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1002/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:47.008058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.009778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.037257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353578d-b131-4cc1-9a20-6baf5c895c1c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.042270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.043508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.043796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.044031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.044465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] PATH_INFO: `/v3/projects/f1af73f8ecf2431fbb426229675f58c5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.045436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1af73f8ecf2431fbb426229675f58c5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.045542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.048222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.060490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ee26b-211c-43bc-8432-4e74500486dc service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.065273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.067889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.067889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.067889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] PATH_INFO: `/v3/users/e8934466baba45ca9540f8667ced8b79` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.068198 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8934466baba45ca9540f8667ced8b79)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.070127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.165286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:47.171255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1af73f8ecf2431fbb426229675f58c5', 'request_id': 'req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.268833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1af73f8ecf2431fbb426229675f58c5', 'request_id': 'req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.278128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1af73f8ecf2431fbb426229675f58c5', 'request_id': 'req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.283324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8934466baba45ca9540f8667ced8b79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.290151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] The token cache is being invalidate because project f1af73f8ecf2431fbb426229675f58c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:47.290531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.291266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.293259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 984/1986] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:47 2022] DELETE /identity/v3/projects/f1af73f8ecf2431fbb426229675f58c5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:47.294094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.295472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.298394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8934466baba45ca9540f8667ced8b79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.356139 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b652-0acf-45af-b7e5-4bbbb8885f4d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.359887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.360533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.360814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.361086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.363034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.408618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8934466baba45ca9540f8667ced8b79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.422263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8934466baba45ca9540f8667ced8b79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.423431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1530c1d6-b63a-47d3-a4ab-279aea55355b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8934466baba45ca9540f8667ced8b79'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:47.434916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1003/1987] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:47 2022] DELETE /identity/v3/users/e8934466baba45ca9540f8667ced8b79 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:47.436339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.437146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945f15f-7b6f-479e-a0a4-614fcb8b11d8 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.437504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945f15f-7b6f-479e-a0a4-614fcb8b11d8 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.437813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3294b8a-39c1-4d9c-bdbc-4ab0b6881a59 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.438146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945f15f-7b6f-479e-a0a4-614fcb8b11d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.449155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 985/1988] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:47.451309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.451761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.452231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.473639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:47.475280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.495575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5945f15f-7b6f-479e-a0a4-614fcb8b11d8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:47.497672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5945f15f-7b6f-479e-a0a4-614fcb8b11d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.533101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:47.585760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 986/1989] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:47.593736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1004/1990] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:47.601871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.603346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.605093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.605401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.605682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.624731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:47.625939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:47.636392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1005/1991] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:47.645071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.654463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40909a-c48d-422e-9bd1-0cccd90652ef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.660128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.660933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.661215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.661489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.662246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:47.662580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.664330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.674188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169e9ca-ce30-44ae-8a1b-ddddbdea3488 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.676666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1006/1992] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:44:47.678190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.678885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.679166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.679441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.681135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.689793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.691092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dd7de-f19b-437d-a548-942dbc18c256 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.691995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.697047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.697840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.698634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.699168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.699981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.703228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:47.708641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 987/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:47.880094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1007/1994] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:47 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:47.884219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:47.893678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a0305-6203-4d57-b6fa-aa633db1921d service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:47.954652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:47.958202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:47.958202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:47.958202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:47.958202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:47.958822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.076451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 988/1995] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:47 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:48.081502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.093479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f2be5-c008-4362-9283-57c267583be2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.100518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.101333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.102186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.102498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.103056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.104693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.116708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1008/1996] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:48.121385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.133326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd982d-f0b5-4f3a-aff5-0a03ab2bc13c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.138447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.139159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.139469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.139743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.140470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.142097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.153954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 989/1997] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:48.158227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.170233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58004230-6edb-4f8c-83e0-0d2229acf399 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.175531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.176605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.176890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.177167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] PATH_INFO: `/v3/projects/a80097997ab4440799a0bc5a1687f6d5/users/f191cb35b84f4f88a34212617b44e3ce/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.192562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80097997ab4440799a0bc5a1687f6d5, user_id=f191cb35b84f4f88a34212617b44e3ce, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.195319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ebff6-5524-4779-b463-3419f3db727c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.267532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.269336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:48.285757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57735279-c630-4bbb-83b0-fadb2072971e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.290427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.291331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.291609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.292490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] PATH_INFO: `/v3/projects/6d53cb44cc53495c87e2ede1b0ccc547` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.294102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d53cb44cc53495c87e2ede1b0ccc547)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.295889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.308279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1009/1998] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:48 2022] PUT /identity/v3/projects/a80097997ab4440799a0bc5a1687f6d5/users/f191cb35b84f4f88a34212617b44e3ce/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:48.321463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.322127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.322599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.354682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] MFA Rules not processed for user `f191cb35b84f4f88a34212617b44e3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:48.355909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:48.422632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1010/1999] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:48.427444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.447102 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d53cb44cc53495c87e2ede1b0ccc547', 'request_id': 'req-eb729a36-e3c3-4592-a969-2c3ad908aea8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:48.495030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ac3da-dd0e-4749-9b22-35a5a77f3814 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.528418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d53cb44cc53495c87e2ede1b0ccc547', 'request_id': 'req-eb729a36-e3c3-4592-a969-2c3ad908aea8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:48.537099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d53cb44cc53495c87e2ede1b0ccc547', 'request_id': 'req-eb729a36-e3c3-4592-a969-2c3ad908aea8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:48.547323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] The token cache is being invalidate because project 6d53cb44cc53495c87e2ede1b0ccc547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:48.547680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:48.548387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:48.550140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.550280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 990/2000] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:48 2022] DELETE /identity/v3/projects/6d53cb44cc53495c87e2ede1b0ccc547 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:48.551631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.551897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.558731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.558731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.558731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.692245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1011/2001] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:48 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:48.696196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.697916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:48.715322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb729a36-e3c3-4592-a969-2c3ad908aea8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.780486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.780486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.780486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.780486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.781055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.782693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.867159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 991/2002] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:48 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:48.871320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.881976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3befe-5741-4895-9410-31500cefddaa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.886889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.887549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.887834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.888106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.888820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.890416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.901386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1012/2003] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:48.909616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.920195 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0640cf8-9928-4b43-826e-ec5f5b158c27 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.924875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.926255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.926544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.926864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.927618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.929199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d673ebc-0b49-4305-a4ba-9ad60b56ed3b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:48.939510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 992/2004] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:48.943706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:48.953671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193998a-296d-469b-b9d0-ebabf1209847 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:48.959657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:48.964077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:48.964077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:48.964077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] PATH_INFO: `/v3/projects/d9149c71ae0148ef8f38ec11ed00b25c/users/772ff188fab54076b0894039fca3f296/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:48.982240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9149c71ae0148ef8f38ec11ed00b25c, user_id=772ff188fab54076b0894039fca3f296, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:48.985681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.060170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1013/2005] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:48 2022] PUT /identity/v3/projects/d9149c71ae0148ef8f38ec11ed00b25c/users/772ff188fab54076b0894039fca3f296/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:49.071395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.071763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.072052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.098715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.100577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:49.102881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] MFA Rules not processed for user `772ff188fab54076b0894039fca3f296`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:49.104074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:49.166362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a92e91-7fcf-456d-b045-3bbe4bed90bf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.171528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 993/2006] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:44:49.171662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.172445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.172726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.175288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] PATH_INFO: `/v3/users/1070f8b2d2e2448fb324a0069546d418` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.175288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1070f8b2d2e2448fb324a0069546d418)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.176772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.177110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.187647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4dc85-9aff-4b0c-984e-c31012b8c6c3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.192635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.193304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.193570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.193846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.194542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.196480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.220103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:49.302961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 994/2007] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:49 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:49.304060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.377112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c4c13-54ed-4210-bc29-9564eb1b55ae admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.382271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.383016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.383300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.383583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.384561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.386663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.445484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1070f8b2d2e2448fb324a0069546d418'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:49.457763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1070f8b2d2e2448fb324a0069546d418'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:49.471790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:49.561001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 995/2008] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:49 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:44:49.562279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.573179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f68787-c64d-4f42-a021-319a6898d14a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.577694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.578390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.578698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.578973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.579698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.581286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.592688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 996/2009] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:49.596706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.602920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1070f8b2d2e2448fb324a0069546d418'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:49.607034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf1add-c544-401d-87c2-31ea5d59e9b7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.611496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1070f8b2d2e2448fb324a0069546d418'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:49.612719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.612889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-63401d45-4f6c-4c0d-aa94-23a9aca81807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1070f8b2d2e2448fb324a0069546d418'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:49.613381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.613736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.614084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] PATH_INFO: `/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a/users/1e43e4350556441da6367a901bc0d6a4/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.622693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1014/2010] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:49 2022] DELETE /identity/v3/users/1070f8b2d2e2448fb324a0069546d418 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:49.624323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.624605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.624956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.631797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3cbd225b54659810f0cf6ebb5ec8a, user_id=1e43e4350556441da6367a901bc0d6a4, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.634877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.645316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:49.646610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:49.710505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1015/2011] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:49.720342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.721800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:49.780179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 997/2012] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:49 2022] PUT /identity/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a/users/1e43e4350556441da6367a901bc0d6a4/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:44:49.781485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.783900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:49.786192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d51276-69ed-4d6e-8380-58caa2bb84ef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.855682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.857065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.857636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.858058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.866302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f141d3e-194f-46d9-8487-bed6694e6fcf admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.871766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.872900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.876180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.876910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.880633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.880633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.880633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.882196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052fd54-74e9-4bf1-b4b2-b4734d3ac1c3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.883228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1016/2013] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:44:49.896735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 998/2014] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:49.899149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:49.910660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9026b-cfce-4266-a396-8ea6e63374bd service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:49.916127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:49.917060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.917344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.917758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] PATH_INFO: `/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a/users/1e43e4350556441da6367a901bc0d6a4/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.920014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f3cbd225b54659810f0cf6ebb5ec8a, user_id=1e43e4350556441da6367a901bc0d6a4, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:49.922938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:49.937069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1017/2015] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:49 2022] PUT /identity/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a/users/1e43e4350556441da6367a901bc0d6a4/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:49.947425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:49.947713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:49.949448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:49.977658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] MFA Rules not processed for user `1e43e4350556441da6367a901bc0d6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:49.982573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:50.059299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 999/2016] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:50.071291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:50.073015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:50.136860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a4d3-56eb-4423-981e-cbb768a6ab2a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:50.140659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:50.141489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:50.141885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:50.142264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:50.145838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:50.209925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:50.212266 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:50.225648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1018/2017] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:50.507583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:50.509840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:50.586170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ea98c-9945-4b28-b79f-024f1ce4142d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:50.590772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:50.591631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:50.591930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:50.592189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] PATH_INFO: `/v3/projects/86404f31d8a24031a73f6aed4db3d9e6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:50.593361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=86404f31d8a24031a73f6aed4db3d9e6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:50.595287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:50.747440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:50.754344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86404f31d8a24031a73f6aed4db3d9e6', 'request_id': 'req-33f42f08-4a1d-457b-ade4-10755ec9fe2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:50.900174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86404f31d8a24031a73f6aed4db3d9e6', 'request_id': 'req-33f42f08-4a1d-457b-ade4-10755ec9fe2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:50.913591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86404f31d8a24031a73f6aed4db3d9e6', 'request_id': 'req-33f42f08-4a1d-457b-ade4-10755ec9fe2d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:50.922174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] The token cache is being invalidate because project 86404f31d8a24031a73f6aed4db3d9e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:50.922464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:50.923439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:50.924987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1000/2018] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:50 2022] DELETE /identity/v3/projects/86404f31d8a24031a73f6aed4db3d9e6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:50.930390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:50.940508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f43249-dbe1-49b2-8b97-eab3cfef8dfe service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:51.002684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:51.003655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:51.004154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:51.004504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] PATH_INFO: `/v3/users/fdeae95c27fa4b14a1512cb196565b3c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:51.005657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdeae95c27fa4b14a1512cb196565b3c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:51.007877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:51.154413 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:51.246426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdeae95c27fa4b14a1512cb196565b3c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:51.258609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdeae95c27fa4b14a1512cb196565b3c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:51.356851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdeae95c27fa4b14a1512cb196565b3c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:51.366039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdeae95c27fa4b14a1512cb196565b3c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:51.366809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-b59289c7-a8b9-4a67-993d-3cef3f859daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdeae95c27fa4b14a1512cb196565b3c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:51.373101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1019/2019] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:50 2022] DELETE /identity/v3/users/fdeae95c27fa4b14a1512cb196565b3c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:51.867114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:51.873604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:51.875143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:51.891289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f42f08-4a1d-457b-ade4-10755ec9fe2d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:51.947433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:51.955287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:51.956182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:51.956482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:51.957907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] PATH_INFO: `/v3/projects/5dba4451411342ff96bed9793c6ae22b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:51.958850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dba4451411342ff96bed9793c6ae22b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:51.960838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.072202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dba4451411342ff96bed9793c6ae22b', 'request_id': 'req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:52.180592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dba4451411342ff96bed9793c6ae22b', 'request_id': 'req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:52.188446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dba4451411342ff96bed9793c6ae22b', 'request_id': 'req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:52.201462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] The token cache is being invalidate because project 5dba4451411342ff96bed9793c6ae22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:44:52.201888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:52.202600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fa3623dd-b4ca-4dcf-a7df-7c6704112d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:52.204179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1001/2020] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:44:51 2022] DELETE /identity/v3/projects/5dba4451411342ff96bed9793c6ae22b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:52.209908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.210383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.210475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.231468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:52.232706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:52.310085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1020/2021] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:52.319178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.319769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.320308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.343023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:52.344230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:52.364714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.369151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1002/2022] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:52.375468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb768-f8d0-4329-968e-2e38a182701b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.383019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.384247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.384475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.384753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.385492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:44:52.385937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.387583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.400250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1021/2023] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:44:52.404594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.413915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dad6a3-1d5a-4e01-abf1-49be416a3a99 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.418540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.419211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.419496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.419772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.420465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.422420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.475944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:52.547245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1003/2024] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:44:52 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:52.551608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.564948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d1ecd-b91d-468f-a233-d092073c0f07 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.626807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.627479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.627765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.628043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.629379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.631365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.792001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1022/2025] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:44:52 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:44:52.793461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.796825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1de5e-e5ca-41db-ba06-5e9f20ef1cff admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.802101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.802802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.803081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.803353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.804089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.805703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.817029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1004/2026] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:52.828344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.838846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a945ff-5617-4bf1-adde-92944795e4e1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.868079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.868079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.868079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.868079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.868554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.868554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b6afd-2eb9-4dde-ad67-66651949ba8a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:52.872404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1023/2027] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:44:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:44:52.876477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:52.882543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:52.887745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005d71d-2170-415e-902a-0f0f61f0002a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:52.892485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:52.893359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:52.893640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:52.893952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] PATH_INFO: `/v3/projects/ee76ebc17e684086b0573655b1d4b6af/users/72bebe9eec964729bc436d023dbdd288/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:52.909563 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee76ebc17e684086b0573655b1d4b6af, user_id=72bebe9eec964729bc436d023dbdd288, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:52.913172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:53.054582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1005/2028] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:44:52 2022] PUT /identity/v3/projects/ee76ebc17e684086b0573655b1d4b6af/users/72bebe9eec964729bc436d023dbdd288/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:44:53.064682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:53.064964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:53.065240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:53.095322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] MFA Rules not processed for user `72bebe9eec964729bc436d023dbdd288`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:53.096535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:53.173388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1024/2029] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:53.187892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:53.189582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:53.250495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24839976-287b-4670-ae43-fc19cbe7fc6d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:53.255474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:53.256120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:53.256398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:53.256695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:53.265203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:53.266721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:53.276261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1006/2030] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:53.463582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:53.465419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:53.511460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:53.540135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c224f6-ebed-40a8-96e2-ccf3876e653c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:53.543968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:53.544642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:53.544910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:53.545185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:53.547060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:53.610321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:53.611804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:53.621754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1025/2031] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:53.727317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:54.803364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:54.805007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:54.818693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748a9dd-035a-45f5-9b33-4e0e28dae585 service glance] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:54.822372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:54.823073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:54.823359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:54.823634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:54.825674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:54.891922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:54.893312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ba1e0-5ba8-442e-875e-58e18b9a235f service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:54.903680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1007/2032] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:54.940054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:56.203894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:56.213545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff216089-21fd-4d88-8cd4-7c9515ccf9bc service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:56.217959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:56.219263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:56.219564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:56.219872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:56.221892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:56.304500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:56.315209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306607b-0fe9-4529-99cd-ca2e93961bb4 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:56.328219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1026/2033] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:59.170935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:59.171483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:59.171483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:59.173128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1008/2034] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:44:59.183914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:59.184433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:59.184664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:59.213537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:44:59.214828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:59.293271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1027/2035] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:44:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:44:59.302796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:59.302894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:59.321586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18797c37-b511-4fa4-ae42-fab1c8f840ee None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:59.327112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:59.327999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:59.328286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:59.328571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:59.343857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:59.345435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:59.357245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:44:59.361258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:44:59.366779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1009/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:44:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:44:59.437362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:44:59.450719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7060234-a131-44b1-8141-750e0b93e914 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:44:59.455384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:44:59.456253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:44:59.456695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:44:59.457088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] PATH_INFO: `/v3/users/a5133bd63bd74551bcecc86dac5ca3d6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:44:59.458317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5133bd63bd74551bcecc86dac5ca3d6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:44:59.460309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:44:59.789976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5133bd63bd74551bcecc86dac5ca3d6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:44:59.802540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5133bd63bd74551bcecc86dac5ca3d6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.133901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5133bd63bd74551bcecc86dac5ca3d6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.142288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5133bd63bd74551bcecc86dac5ca3d6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.142973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5133bd63bd74551bcecc86dac5ca3d6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.151550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1028/2037] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:44:59 2022] DELETE /identity/v3/users/a5133bd63bd74551bcecc86dac5ca3d6 => generated 0 bytes in 796 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:00.415759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:00.429320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef684-94ea-4cbd-8af1-9eedee9a53b9 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:00.484406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:00.513714 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:00.514641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:00.514922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:00.515342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] PATH_INFO: `/v3/projects/9ec6517616c94b799e70612816bf9d76` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:00.516571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec6517616c94b799e70612816bf9d76)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:00.518455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:00.537706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:00.537706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:00.552208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631d5e9-5ae4-4102-b4d5-99c67c0a3f86 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:00.558116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:00.563048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:00.563048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:00.563048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] PATH_INFO: `/v3/users/0bfcd00d9a2c41e38c9084aa65a718fd` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:00.563048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfcd00d9a2c41e38c9084aa65a718fd)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:00.563048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:00.661695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec6517616c94b799e70612816bf9d76', 'request_id': 'req-7e4ba93a-a39b-407a-8392-96da82e97f49'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.743429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec6517616c94b799e70612816bf9d76', 'request_id': 'req-7e4ba93a-a39b-407a-8392-96da82e97f49'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.751358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec6517616c94b799e70612816bf9d76', 'request_id': 'req-7e4ba93a-a39b-407a-8392-96da82e97f49'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.762562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] The token cache is being invalidate because project 9ec6517616c94b799e70612816bf9d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:00.762860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.763643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7e4ba93a-a39b-407a-8392-96da82e97f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.765267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1010/2038] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:00 2022] DELETE /identity/v3/projects/9ec6517616c94b799e70612816bf9d76 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:00.769886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825852e-2f91-4be3-b77e-f01605da6d58 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:00.770125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825852e-2f91-4be3-b77e-f01605da6d58 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:00.770744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825852e-2f91-4be3-b77e-f01605da6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:00.793700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d825852e-2f91-4be3-b77e-f01605da6d58 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:00.794959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d825852e-2f91-4be3-b77e-f01605da6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:00.795783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfcd00d9a2c41e38c9084aa65a718fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.812605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfcd00d9a2c41e38c9084aa65a718fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.868034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1011/2039] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:00.875604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:00.875897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:00.876177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:00.904259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:00.905846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:00.911951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfcd00d9a2c41e38c9084aa65a718fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.916498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1012/2040] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:00.920615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:00.924234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfcd00d9a2c41e38c9084aa65a718fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.924941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d7180381-cbf6-4f38-bacc-da55e937d2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfcd00d9a2c41e38c9084aa65a718fd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:00.931238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019a858-302e-4ad3-9989-f96235b9ea54 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:00.932983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1029/2041] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:00 2022] DELETE /identity/v3/users/0bfcd00d9a2c41e38c9084aa65a718fd => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:00.942144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:00.942433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:00.942739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:00.964867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:00.966628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:01.019886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:01.020812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.021580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.021866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.022140 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.022927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:01.036209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.036209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.041091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1013/2042] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:01.042416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1030/2043] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:01.045411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.047108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:01.053067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.054622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:01.060697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d5fac-e8c0-456e-9b74-8ef355548988 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.065454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.066217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.066481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.066787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.067478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.069388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.128176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f621be-39ae-45f9-9b04-826e8fc7c91b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.132498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.133150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.133429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.133698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.139630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:01.143752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.148296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.156191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1031/2044] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:01.178155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1014/2045] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:01 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:01.182324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.193754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e7751-6ffc-48ff-94f5-c294146eaa6b service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.273688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.273688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.273941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.274288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.275452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.277351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.417373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1032/2046] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:01 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:01.418632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.434499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c02c0-18b6-49d0-9d7d-5447b193f9d1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.439129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.439785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.440069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.440388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.441788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.443429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.458130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1015/2047] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:01.461286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.473364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000212-23e6-4420-ae58-68ca97a6571a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.479503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.480247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.480711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.481025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.481762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.483553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a76694-d2d3-4edb-9be6-969bde74f0f6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.494498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1033/2048] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:01.499331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.509858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871bac5-d37f-4e18-a725-17c90051739d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.514653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.515616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.515929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.516206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] PATH_INFO: `/v3/projects/563016cfa9f941e9ad9f8e2fecf6a934/users/6e21261a1a42499cba2df02ed2466e46/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.534723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=563016cfa9f941e9ad9f8e2fecf6a934, user_id=6e21261a1a42499cba2df02ed2466e46, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.538570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.700891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1016/2049] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:01 2022] PUT /identity/v3/projects/563016cfa9f941e9ad9f8e2fecf6a934/users/6e21261a1a42499cba2df02ed2466e46/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:01.711807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.712295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.712409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.728968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.731052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:01.747072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] MFA Rules not processed for user `6e21261a1a42499cba2df02ed2466e46`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:01.748305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:01.808080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e0fe0-0c94-4283-b6c9-794556e15cc7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.813332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.814375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.814747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.815021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] PATH_INFO: `/v3/projects/b201a29a1d934b6aa719fe6e2e0eb713` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.816473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b201a29a1d934b6aa719fe6e2e0eb713)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.818823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.818959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:01.828915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1034/2050] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:01.833289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:01.844139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6cce3-427a-4331-af0e-c80717f598ce None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:01.849613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:01.850286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:01.850606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:01.850879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:01.851567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:01.853734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:01.970767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b201a29a1d934b6aa719fe6e2e0eb713', 'request_id': 'req-8e17a59f-a3c6-4f01-8330-932a06b9e53e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:01.979874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1035/2051] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:01 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:01.980811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.043704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fade97-27db-422b-96d8-bcf5d2184d44 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.048451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.049112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.049395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.049672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.050682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.052905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.057272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b201a29a1d934b6aa719fe6e2e0eb713', 'request_id': 'req-8e17a59f-a3c6-4f01-8330-932a06b9e53e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.064725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b201a29a1d934b6aa719fe6e2e0eb713', 'request_id': 'req-8e17a59f-a3c6-4f01-8330-932a06b9e53e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.076882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] The token cache is being invalidate because project b201a29a1d934b6aa719fe6e2e0eb713 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:02.077168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.077827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.079929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1017/2052] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:01 2022] DELETE /identity/v3/projects/b201a29a1d934b6aa719fe6e2e0eb713 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:02.084328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.085718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.119240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:02.154161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1036/2053] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:01 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:02.154379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17a59f-a3c6-4f01-8330-932a06b9e53e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.155454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.157318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.158938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.160609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.160888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.161161 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] PATH_INFO: `/v3/users/90246be1fd2849bb941dd3166e2985a1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.162356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=90246be1fd2849bb941dd3166e2985a1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.164305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.169915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1e24b-a78a-4124-bbd2-3469c5a0e727 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.174807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.175676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.176011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.176294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.177208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.179005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.193156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1037/2054] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:02.194391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.204501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e5afe-83f1-4e47-ba0e-2156592261e1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.209944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.210843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.211128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.211405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] PATH_INFO: `/v3/projects/d1e988eca03b47e19f4682c129933f0d/users/3b85dc2ccc3c4fc49d046799c8cc67b1/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.228412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e988eca03b47e19f4682c129933f0d, user_id=3b85dc2ccc3c4fc49d046799c8cc67b1, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.231180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.346188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1038/2055] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:02 2022] PUT /identity/v3/projects/d1e988eca03b47e19f4682c129933f0d/users/3b85dc2ccc3c4fc49d046799c8cc67b1/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:02.348221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.350912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.390791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90246be1fd2849bb941dd3166e2985a1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.419200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90246be1fd2849bb941dd3166e2985a1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.447171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5299d-a74b-4bbe-a28a-34cb21be9d0e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.452048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.453122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.453276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.453553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] PATH_INFO: `/v3/users/f191cb35b84f4f88a34212617b44e3ce` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.454830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f191cb35b84f4f88a34212617b44e3ce)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.458062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.522240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90246be1fd2849bb941dd3166e2985a1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.529656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90246be1fd2849bb941dd3166e2985a1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.530304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90246be1fd2849bb941dd3166e2985a1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.536428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1018/2056] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:02 2022] DELETE /identity/v3/users/90246be1fd2849bb941dd3166e2985a1 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:02.537209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.538831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.608827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01826cb2-7435-44c4-b884-264243a86f9b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.613869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.614831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.615112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.615402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.616116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.617834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.632291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1019/2057] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:02.633320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.650219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1b6ca-2598-46b8-870b-f7aa6a987fdc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.655312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.656341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.656631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.656910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] PATH_INFO: `/v3/projects/d1e988eca03b47e19f4682c129933f0d/users/3b85dc2ccc3c4fc49d046799c8cc67b1/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.660013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e988eca03b47e19f4682c129933f0d, user_id=3b85dc2ccc3c4fc49d046799c8cc67b1, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.662717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648dcdb-0b56-4ca9-8235-c9d0556a1e43 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.673491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1020/2058] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:02 2022] PUT /identity/v3/projects/d1e988eca03b47e19f4682c129933f0d/users/3b85dc2ccc3c4fc49d046799c8cc67b1/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:02.683848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.684141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.684418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.715531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] MFA Rules not processed for user `3b85dc2ccc3c4fc49d046799c8cc67b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:02.716782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.734611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:02.763327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:02.777347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f191cb35b84f4f88a34212617b44e3ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.790315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f191cb35b84f4f88a34212617b44e3ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.793040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1021/2059] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:02.803755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:02.805146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:02.865926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9c166-7c95-4283-bdb6-60d64575242a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:02.877426 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:02.878800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.879192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.879562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.893388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:02.895476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5420b2b-3633-4b83-b3bf-1f2d6b8ed3cb service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:02.899855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f191cb35b84f4f88a34212617b44e3ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.913726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f191cb35b84f4f88a34212617b44e3ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.913726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f191cb35b84f4f88a34212617b44e3ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:02.913872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1022/2060] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:02.917350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1039/2061] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:02 2022] DELETE /identity/v3/users/f191cb35b84f4f88a34212617b44e3ce => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:02.925618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:02.925618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:02.925618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:02.947062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:02.948298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:03.016122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1023/2062] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:03.026310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:03.027820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:03.097662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cac792-d2e6-4ec6-9c02-36481fccfe03 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:03.108872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:03.108872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:03.108872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:03.108872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:03.114363 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:03.115841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:03.125654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1040/2063] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:03.166668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:03.166668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:03.180022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b852e7-fcff-4cf1-b4d6-d39c870aeb86 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:03.197886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:03.199026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:03.199142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:03.199412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] PATH_INFO: `/v3/projects/b343f924d73b4d1cb8a633ecf8c0403c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:03.200773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b343f924d73b4d1cb8a633ecf8c0403c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:03.202711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:03.337179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b343f924d73b4d1cb8a633ecf8c0403c', 'request_id': 'req-8184dd70-ede6-44b1-a063-32051cdba883'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.340090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:03.424854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b343f924d73b4d1cb8a633ecf8c0403c', 'request_id': 'req-8184dd70-ede6-44b1-a063-32051cdba883'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.435379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b343f924d73b4d1cb8a633ecf8c0403c', 'request_id': 'req-8184dd70-ede6-44b1-a063-32051cdba883'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.449947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] The token cache is being invalidate because project b343f924d73b4d1cb8a633ecf8c0403c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:03.449947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.450729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.453097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1024/2064] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:03 2022] DELETE /identity/v3/projects/b343f924d73b4d1cb8a633ecf8c0403c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:03.458252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:03.471187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a34f57-0fef-4900-873f-d7fb1f01a610 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:03.571403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:03.572464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:03.572708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:03.572984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] PATH_INFO: `/v3/users/ae94aea892594ad08c5c7826ed673769` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:03.574582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae94aea892594ad08c5c7826ed673769)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:03.577684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:03.719264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:03.721239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:03.736618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184dd70-ede6-44b1-a063-32051cdba883 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:03.749367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:03.750802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:03.762148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:03.762999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] PATH_INFO: `/v3/projects/a80097997ab4440799a0bc5a1687f6d5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:03.764561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80097997ab4440799a0bc5a1687f6d5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:03.766658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:03.883111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae94aea892594ad08c5c7826ed673769'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.887513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:03.900635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae94aea892594ad08c5c7826ed673769'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:03.915475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80097997ab4440799a0bc5a1687f6d5', 'request_id': 'req-312ba902-1487-4469-9163-32f32fbdc277'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.002296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae94aea892594ad08c5c7826ed673769'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.010722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae94aea892594ad08c5c7826ed673769'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.011385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae94aea892594ad08c5c7826ed673769'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.017303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1041/2065] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:03 2022] DELETE /identity/v3/users/ae94aea892594ad08c5c7826ed673769 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:04.078089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:04.079596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.128669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80097997ab4440799a0bc5a1687f6d5', 'request_id': 'req-312ba902-1487-4469-9163-32f32fbdc277'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.137171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80097997ab4440799a0bc5a1687f6d5', 'request_id': 'req-312ba902-1487-4469-9163-32f32fbdc277'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.149636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] The token cache is being invalidate because project a80097997ab4440799a0bc5a1687f6d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:04.149930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.150645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.152204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1025/2066] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:03 2022] DELETE /identity/v3/projects/a80097997ab4440799a0bc5a1687f6d5 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:04.155701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291da76-08e3-4c8c-be64-548d4eceae28 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:04.156073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:04.157353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.162559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:04.162559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.162868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.163188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] PATH_INFO: `/v3/users/6e21261a1a42499cba2df02ed2466e46` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.165360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e21261a1a42499cba2df02ed2466e46)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:04.167356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:04.228364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba902-1487-4469-9163-32f32fbdc277 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:04.232909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:04.233777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.234060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.234336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] PATH_INFO: `/v3/users/772ff188fab54076b0894039fca3f296` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.235682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=772ff188fab54076b0894039fca3f296)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:04.238160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:04.390182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e21261a1a42499cba2df02ed2466e46'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.402866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e21261a1a42499cba2df02ed2466e46'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.455390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772ff188fab54076b0894039fca3f296'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.468738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772ff188fab54076b0894039fca3f296'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.470491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=19,keystone:SELECT=80,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:04.551460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e21261a1a42499cba2df02ed2466e46'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.560090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e21261a1a42499cba2df02ed2466e46'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.560781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e21261a1a42499cba2df02ed2466e46'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.568808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1042/2067] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:04 2022] DELETE /identity/v3/users/6e21261a1a42499cba2df02ed2466e46 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:04.571674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:04.571674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.640933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772ff188fab54076b0894039fca3f296'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.643321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee2066-effe-4251-b1f5-e60e99e2ffd9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:04.653657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:04.655091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772ff188fab54076b0894039fca3f296'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.655233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.655557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.656028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] PATH_INFO: `/v3/projects/7d23ed3f65434d3a952544b856cb6974` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.656309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-56d22bba-0232-4c03-b906-3f9151393c3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772ff188fab54076b0894039fca3f296'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.657964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d23ed3f65434d3a952544b856cb6974)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:04.661511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:04.666241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1026/2068] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:04 2022] DELETE /identity/v3/users/772ff188fab54076b0894039fca3f296 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:04.669340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.669340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.669486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.697723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:04.700307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.779537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d23ed3f65434d3a952544b856cb6974', 'request_id': 'req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.791205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d23ed3f65434d3a952544b856cb6974', 'request_id': 'req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.800753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1027/2069] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:04.810980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:04.812625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.871627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d23ed3f65434d3a952544b856cb6974', 'request_id': 'req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.876994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0738da6-11f5-480e-a4e5-e02efcb1db09 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:04.890172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] The token cache is being invalidate because project 7d23ed3f65434d3a952544b856cb6974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:04.893055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.893055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8bbb162-cedc-4ed9-8f1b-6f8f9903af42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:04.893229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1043/2070] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:04 2022] DELETE /identity/v3/projects/7d23ed3f65434d3a952544b856cb6974 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:04.900231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d2e5f-2572-4f67-93c6-e32f39ffb946 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.901147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d2e5f-2572-4f67-93c6-e32f39ffb946 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.901605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d2e5f-2572-4f67-93c6-e32f39ffb946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.927900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ff6d2e5f-2572-4f67-93c6-e32f39ffb946 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:04.931659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ff6d2e5f-2572-4f67-93c6-e32f39ffb946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.945648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:04.946301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:04.946855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:04.946855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:04.948796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:04.968066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:05.009700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1044/2071] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:05.018829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.019154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.019660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.031181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.032651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.039240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:05.040467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:05.047650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1028/2072] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:05.050954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1045/2073] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:05.055341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.065386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f790a1-cb38-4b0f-866e-a427d8b4bd79 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.071044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.071718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.072028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.072304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.073033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:05.073339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.074966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.086765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1029/2074] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:05.091562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.101392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa839-9fe7-47c2-b2b3-e790ca1863c7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.106471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.107309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.107596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.107875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.108571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.110550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.240235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.242754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:05.257393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b38f6f-94ce-4ac2-88dc-6bc58f1d272d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.263059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.264106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.264285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.264473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] PATH_INFO: `/v3/projects/d9149c71ae0148ef8f38ec11ed00b25c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.265690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9149c71ae0148ef8f38ec11ed00b25c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.267914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.274415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1046/2075] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:05 2022] POST /identity/v3/projects => generated 393 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:05.275519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.353861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7004a-67b4-4957-a76d-bacee9176e5b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.360336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.361017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.361299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.361830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.363068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.366213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.423623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9149c71ae0148ef8f38ec11ed00b25c', 'request_id': 'req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:05.485815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1047/2076] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:05 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:05.486921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.498373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717401d-9cdd-476b-b44f-d598f82d793d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.505238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9149c71ae0148ef8f38ec11ed00b25c', 'request_id': 'req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:05.514491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9149c71ae0148ef8f38ec11ed00b25c', 'request_id': 'req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:05.532607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] The token cache is being invalidate because project d9149c71ae0148ef8f38ec11ed00b25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:05.532984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:05.546398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:05.554993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1030/2077] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:05 2022] DELETE /identity/v3/projects/d9149c71ae0148ef8f38ec11ed00b25c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:05.556085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.563597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:05.577518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.579231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.579516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.580246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.582410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:05.582552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.583491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.594233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1048/2078] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:05.598457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.600514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:05.615401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d40539-1f46-4891-8e07-0b299250691a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.620303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.621019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.621241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.621518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.622235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.623872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.636322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1049/2079] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:05.640586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.651144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835c92d-f399-456a-b2b3-36ab3693179a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.656158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.657027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.657532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.657654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] PATH_INFO: `/v3/projects/31ac0d64958446cbbafdb4d1d5970081/users/1f9f660da5504605b32b2e8c672df04b/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.694387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ac0d64958446cbbafdb4d1d5970081, user_id=1f9f660da5504605b32b2e8c672df04b, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.696691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.712561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54c7c6-a516-4306-aa3d-a57d74f1d43c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.717275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.718134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.718411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.718710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] PATH_INFO: `/v3/users/1e43e4350556441da6367a901bc0d6a4` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.719895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e43e4350556441da6367a901bc0d6a4)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.722475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:05.821689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1050/2080] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:05 2022] PUT /identity/v3/projects/31ac0d64958446cbbafdb4d1d5970081/users/1f9f660da5504605b32b2e8c672df04b/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:05.822570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:05.824393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:05.896140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5e5ee-057b-477d-be6d-65dd0295376d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:05.901387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:05.902281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:05.902724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:05.902938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] PATH_INFO: `/v3/projects/563016cfa9f941e9ad9f8e2fecf6a934` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:05.904201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=563016cfa9f941e9ad9f8e2fecf6a934)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:05.906308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.001313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e43e4350556441da6367a901bc0d6a4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.013700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e43e4350556441da6367a901bc0d6a4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.093411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '563016cfa9f941e9ad9f8e2fecf6a934', 'request_id': 'req-d8639481-3a66-4b57-80f5-88077c809c55'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.099829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '563016cfa9f941e9ad9f8e2fecf6a934', 'request_id': 'req-d8639481-3a66-4b57-80f5-88077c809c55'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.105072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e43e4350556441da6367a901bc0d6a4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.112696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e43e4350556441da6367a901bc0d6a4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.113427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9998fb57-3885-4dd1-b9af-10d6cf78dc7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e43e4350556441da6367a901bc0d6a4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.119745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1031/2081] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:05 2022] DELETE /identity/v3/users/1e43e4350556441da6367a901bc0d6a4 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:06.128693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.129022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.129303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.156967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] MFA Rules not processed for user `1f9f660da5504605b32b2e8c672df04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:06.158178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.214859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '563016cfa9f941e9ad9f8e2fecf6a934', 'request_id': 'req-d8639481-3a66-4b57-80f5-88077c809c55'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.226509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1032/2082] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:06.236940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] The token cache is being invalidate because project 563016cfa9f941e9ad9f8e2fecf6a934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:06.236940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.236940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.237223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.237314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1051/2083] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:05 2022] DELETE /identity/v3/projects/563016cfa9f941e9ad9f8e2fecf6a934 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:06.237412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.238094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.240134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.248205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:06.303315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92714860-e897-4f24-9985-bb5ac9557121 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:06.308293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8639481-3a66-4b57-80f5-88077c809c55 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:06.308494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:06.309261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.309523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.310080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.310487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:06.312458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.314656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:06.315602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.316894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.316894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] PATH_INFO: `/v3/users/3b85dc2ccc3c4fc49d046799c8cc67b1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.318943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b85dc2ccc3c4fc49d046799c8cc67b1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:06.320852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.399784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1033/2084] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:06 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:06.408381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.408381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.471398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500517d7-08ea-4dc0-94d9-8c5132b6bc1b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:06.477053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:06.477725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.478006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.478294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.479017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:06.480929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.575873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b85dc2ccc3c4fc49d046799c8cc67b1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.582248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1034/2085] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:06 2022] POST /identity/v3/users => generated 302 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:06.583242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.585327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.589722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b85dc2ccc3c4fc49d046799c8cc67b1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.599672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeee6e2-50d8-4c10-86bd-0031602f145c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:06.606257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:06.607420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.607859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.608232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] PATH_INFO: `/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.610080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f3cbd225b54659810f0cf6ebb5ec8a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:06.612152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.814901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f3cbd225b54659810f0cf6ebb5ec8a', 'request_id': 'req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.865117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b85dc2ccc3c4fc49d046799c8cc67b1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.872588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b85dc2ccc3c4fc49d046799c8cc67b1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.873237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b85dc2ccc3c4fc49d046799c8cc67b1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.879982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1052/2086] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:06 2022] DELETE /identity/v3/users/3b85dc2ccc3c4fc49d046799c8cc67b1 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:06.881548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.882956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.897186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:06.921822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f3cbd225b54659810f0cf6ebb5ec8a', 'request_id': 'req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.929826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f3cbd225b54659810f0cf6ebb5ec8a', 'request_id': 'req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.942934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] The token cache is being invalidate because project 81f3cbd225b54659810f0cf6ebb5ec8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:06.943227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.943899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2a5e43c9-5e32-4fdc-b654-43fe9fc4fda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:06.945482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1035/2087] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:06 2022] DELETE /identity/v3/projects/81f3cbd225b54659810f0cf6ebb5ec8a => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:06.951257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109efc6e-6984-46f1-835c-62367a5a8d6d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.958268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109efc6e-6984-46f1-835c-62367a5a8d6d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.958268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109efc6e-6984-46f1-835c-62367a5a8d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.958478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3de23f-929f-42ae-b4f7-a9d3853d1905 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:06.962752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:06.963661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:06.964157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:06.964643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:06.965588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:06.967512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:06.978144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1053/2088] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:06.982325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:06.984191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.988270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-109efc6e-6984-46f1-835c-62367a5a8d6d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:06.989925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-109efc6e-6984-46f1-835c-62367a5a8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:06.997695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10856a0a-43eb-450b-bdc1-f223cf70c0fb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.005760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1036/2089] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:07.005940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.005940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.005940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.005940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] PATH_INFO: `/v3/projects/c36db6691b6d4dd2bfee7559f6218381/users/8e97798fcb8e48369731981e75a2594c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.008103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.008401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.008689 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.020895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36db6691b6d4dd2bfee7559f6218381, user_id=8e97798fcb8e48369731981e75a2594c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.023614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.029186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:07.030724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.039515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1037/2090] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:07.044493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.053377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacab3ec-5fd0-4406-a640-6b4225137801 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.058118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.058801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.059088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.059366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.060149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:07.060462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.062184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.079574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1038/2091] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:07.080858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.092271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bd572-5554-405c-829c-b64af7dcb761 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.099864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.101337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.101687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.102033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.102844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.107559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.111271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1054/2092] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:06 2022] PUT /identity/v3/projects/c36db6691b6d4dd2bfee7559f6218381/users/8e97798fcb8e48369731981e75a2594c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:07.115767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.173873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:07.214937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d22ac82-71b2-4633-9451-2e1ec7f063f6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.219823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1039/2093] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:07 2022] POST /identity/v3/projects => generated 399 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:07.223774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.227614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.228314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.228599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.228878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.229619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.231746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.247449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1055/2094] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:07.262300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.274119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ad0a2-8bd2-4172-a1c7-b769bf6e10b5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.302334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794edf2-ab69-486b-b957-ab4efefc6cb2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.308186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.309101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.309385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.309669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.310373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.312360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.335374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.336368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.336653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.336930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] PATH_INFO: `/v3/projects/c36db6691b6d4dd2bfee7559f6218381/users/8e97798fcb8e48369731981e75a2594c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.340096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36db6691b6d4dd2bfee7559f6218381, user_id=8e97798fcb8e48369731981e75a2594c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.341650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43a52ec-09b0-477b-972a-ded7a41a9690 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.352924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1056/2095] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:07 2022] PUT /identity/v3/projects/c36db6691b6d4dd2bfee7559f6218381/users/8e97798fcb8e48369731981e75a2594c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:07.365085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.365387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.365707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.399651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] MFA Rules not processed for user `8e97798fcb8e48369731981e75a2594c`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:07.401068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.432009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1040/2096] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:07 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:07.433046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.434872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.448659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55abeca-3b29-46ab-85d3-9228b7754e87 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.501259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1057/2097] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:07.502475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.514883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.515780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.516108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.516382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] PATH_INFO: `/v3/projects/d1e988eca03b47e19f4682c129933f0d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.517569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e988eca03b47e19f4682c129933f0d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.519789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.557323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:07.576166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bde20a-493f-43d1-be65-69b662c3ae3e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.580842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.581510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.581794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.582112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.583702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.584887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.594876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1058/2098] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:07.595851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.598059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.636987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e988eca03b47e19f4682c129933f0d', 'request_id': 'req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:07.658306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfc56e-70bb-476d-ae53-2fd8ced6c1d9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.707530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.708185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.708467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.708743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.773275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e988eca03b47e19f4682c129933f0d', 'request_id': 'req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:07.777559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.779145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.781427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e988eca03b47e19f4682c129933f0d', 'request_id': 'req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:07.790488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1059/2099] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:07.791440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.794485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] The token cache is being invalidate because project d1e988eca03b47e19f4682c129933f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:07.794807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:07.795488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-83f78e2c-e5ad-49c6-ae33-44b9523cdf43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:07.797863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1041/2100] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:07 2022] DELETE /identity/v3/projects/d1e988eca03b47e19f4682c129933f0d => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:07.803340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.803500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.803846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.823137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:07.824482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.856830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d1fc-188d-4c71-adfd-c36c09bb038b service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.862082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.864372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.864372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.865065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.866435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.867890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.885643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1060/2101] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:07.887478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.889412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.896340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1042/2102] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:07.908708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:07.908708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.918427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6378-cf2d-4f53-9ec2-cecb2b81abfe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.922426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.923113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.923401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.923680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.925607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:07.934160 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11686ce9-4094-4cfc-9a7d-6e7d0d157aeb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:07.940054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:07.940946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:07.941234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:07.941523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] PATH_INFO: `/v3/projects/0993ce09e1dd44a09c842964cae7948a/users/d282fe306d7d4927bdfe7ced672dab64/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:07.957790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0993ce09e1dd44a09c842964cae7948a, user_id=d282fe306d7d4927bdfe7ced672dab64, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.960783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df3f26-cb62-46d1-ae37-cf1cfa401d9e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:07.990351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:07.991852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605bebb-37b3-4b62-b4a2-f1ecf61b79d1 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.000727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1061/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:08.001976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.002259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.002562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.055006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:08.056438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:08.111659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1062/2104] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:08.117865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.124171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1043/2105] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:07 2022] PUT /identity/v3/projects/0993ce09e1dd44a09c842964cae7948a/users/d282fe306d7d4927bdfe7ced672dab64/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:08.128117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdeabf3-f8dd-4e12-abf2-11009e3d09fb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.134920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.135478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.135852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.164387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] MFA Rules not processed for user `d282fe306d7d4927bdfe7ced672dab64`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:08.165525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:08.188746 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.189459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.189736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.193194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.193194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:08.193194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.193194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.206561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1063/2106] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:08.212757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.214320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:08.227494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317e575-affe-4548-b764-6dd42f002aef admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.233391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.234057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.234336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.234639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.235329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.237243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.237595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1044/2107] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:08.291630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.294367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:08.373463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1064/2108] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:08 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:08.375992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61cfa-9c6e-4010-84a0-057b77effb97 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.382244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.435244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.436028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.436312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.436710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.456454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861783c-1d79-4e13-90ae-19e7c7cea2d2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.463414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.463945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.464328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.464712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.465506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.468262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.498733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.505826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.509759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1045/2109] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:08.580594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1065/2110] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:08 2022] POST /identity/v3/users => generated 283 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:08.586620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.591679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:08.595973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f993212-9c59-47da-9ebc-dbbae6e8ef4c service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.600829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.601616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.601980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.602329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.603163 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.604872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.615890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1046/2111] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:08.620056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.631119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6ce7b-71fb-4067-83ce-462d2cf71f71 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.636432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.637099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.637511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.637794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.638582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.640212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a09c5-e67a-43f9-80d6-982f97540462 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.654576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1066/2112] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:08.658838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:08.669940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c810f-a1c9-48c6-b0d7-547002846aaa admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:08.674401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:08.675394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.676881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.676881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] PATH_INFO: `/v3/projects/109ba3059eb247d99e41eaf8b09cb5a7/users/0afdd60ae5a1444095657f5ce9d23fe5/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.693210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=109ba3059eb247d99e41eaf8b09cb5a7, user_id=0afdd60ae5a1444095657f5ce9d23fe5, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:08.695925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:08.888225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1047/2113] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:08 2022] PUT /identity/v3/projects/109ba3059eb247d99e41eaf8b09cb5a7/users/0afdd60ae5a1444095657f5ce9d23fe5/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:08.900021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:08.900315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:08.900610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:08.928662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] MFA Rules not processed for user `0afdd60ae5a1444095657f5ce9d23fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:08.929882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:09.001074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1067/2114] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:09.006990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.071124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c08d2-9f46-4e0f-ae8e-1a88e72f8950 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.075907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.076628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.076884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.077163 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.077861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.079855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.170008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1048/2115] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:09 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:09.174293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.186262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce290ea7-5d2a-496b-8cc8-c91e4572a0ef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.246706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.247383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.247671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.247948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.248643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.250636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.254706 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:09.369815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1068/2116] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:09 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:09.373272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.384066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705218f7-913b-445b-b44e-1414fd91c343 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.388801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.389639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.389939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.390220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.390989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.392774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.403680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1049/2117] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:09.407849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.418809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0260b-07cb-4419-8168-13777c823cb1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.423500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.424440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.424666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.424954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] PATH_INFO: `/v3/projects/d69b1345669e410c8d6922787e46110a/users/9a54d638f60a4084a0e0925d8f93bb15/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.440694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69b1345669e410c8d6922787e46110a, user_id=9a54d638f60a4084a0e0925d8f93bb15, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.444657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.533433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1069/2118] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:09 2022] PUT /identity/v3/projects/d69b1345669e410c8d6922787e46110a/users/9a54d638f60a4084a0e0925d8f93bb15/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:09.538323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.548454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed668c3-dca3-46ff-b9e4-01d59ede8aaf admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.606169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.606872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.607162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.607445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.608175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.609806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bd5c2-ffcc-4baf-a81e-2b9d3b50df1c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.623641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1050/2119] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:09.624892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.635402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183de2e7-7bb6-40c0-8f94-9455da60dbc6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.641324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.642270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.642578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.644035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] PATH_INFO: `/v3/projects/d69b1345669e410c8d6922787e46110a/users/9a54d638f60a4084a0e0925d8f93bb15/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.645435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69b1345669e410c8d6922787e46110a, user_id=9a54d638f60a4084a0e0925d8f93bb15, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.648180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.658992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1070/2120] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:09 2022] PUT /identity/v3/projects/d69b1345669e410c8d6922787e46110a/users/9a54d638f60a4084a0e0925d8f93bb15/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:09.674948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.675477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.675477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.701395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] MFA Rules not processed for user `9a54d638f60a4084a0e0925d8f93bb15`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:09.702646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:09.732777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:09.771923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1051/2121] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:09.783347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:09.840395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344d41b-baac-4045-bbd5-8aeaee0dc179 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:09.844267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:09.844920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:09.845202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:09.845480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:09.855668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:09.856595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:09.867315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1071/2122] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:10.087633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:10.135294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:10.144944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b8d73b-8455-41f4-ab8d-8538a63b186b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:10.149023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:10.149661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:10.149941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:10.150215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:10.207796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:10.209156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41efd92b-d69d-42a0-ae28-6d036899a1de service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:10.218129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1052/2123] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:10.376659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:10.571482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:10.697320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:11.059246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:20.966076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:21.334009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=56 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:27.292314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:27.293808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:27.361472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d688d-fd7b-4bc8-804f-5e5d3e76a9bd service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:27.367387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:27.368899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:27.369189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:27.369493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] PATH_INFO: `/v3/users/1f9f660da5504605b32b2e8c672df04b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:27.370857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f9f660da5504605b32b2e8c672df04b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:27.372794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:27.602433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f9f660da5504605b32b2e8c672df04b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:27.615566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f9f660da5504605b32b2e8c672df04b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:27.761559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f9f660da5504605b32b2e8c672df04b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:27.771941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f9f660da5504605b32b2e8c672df04b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:27.772679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f9f660da5504605b32b2e8c672df04b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:27.783484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1072/2124] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:27 2022] DELETE /identity/v3/users/1f9f660da5504605b32b2e8c672df04b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:27.785231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:27.785713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:27.785952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:27.806702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:27.808057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:27.873225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1053/2125] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:27.882975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:27.885270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:27.967256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d863791f-86d3-4cca-bfcc-c32465078c65 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:27.971722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:27.972374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:27.972652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:27.972923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:27.982945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:27.987837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:27.995030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1073/2126] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:28.299649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:28.301175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.313584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03feceb3-0a0f-443c-8eb7-e952650c168a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:28.318365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:28.319355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:28.319700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:28.319953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] PATH_INFO: `/v3/users/d282fe306d7d4927bdfe7ced672dab64` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:28.326375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d282fe306d7d4927bdfe7ced672dab64)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:28.326375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:28.519636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:28.521314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.531158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:28.537304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975bcaa-d9f0-413d-9425-91cadb3efa96 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:28.542978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:28.543826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:28.544116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:28.544420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] PATH_INFO: `/v3/projects/31ac0d64958446cbbafdb4d1d5970081` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:28.545627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ac0d64958446cbbafdb4d1d5970081)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:28.547740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:28.571833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd282fe306d7d4927bdfe7ced672dab64'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.584656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd282fe306d7d4927bdfe7ced672dab64'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.692290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd282fe306d7d4927bdfe7ced672dab64'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.702355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd282fe306d7d4927bdfe7ced672dab64'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.703058 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd089ab5-9736-43ac-a1cf-e7255a02ac66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd282fe306d7d4927bdfe7ced672dab64'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.709437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1054/2127] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:28 2022] DELETE /identity/v3/users/d282fe306d7d4927bdfe7ced672dab64 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:28.711021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ac0d64958446cbbafdb4d1d5970081', 'request_id': 'req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.713939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:28.714246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:28.714591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:28.719327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ac0d64958446cbbafdb4d1d5970081', 'request_id': 'req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.734488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:28.735820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.798584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1055/2128] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:28.808395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:28.809775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.824555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:28.852528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ac0d64958446cbbafdb4d1d5970081', 'request_id': 'req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.866018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] The token cache is being invalidate because project 31ac0d64958446cbbafdb4d1d5970081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:28.866452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.867163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:28.869368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1074/2129] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:28 2022] DELETE /identity/v3/projects/31ac0d64958446cbbafdb4d1d5970081 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:28.873019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:28.874594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.877180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93255b08-ab9a-4705-817f-fc3868135e13 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:28.881679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:28.882555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:28.883016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:28.883977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:28.885919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:28.898096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:28.904030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:28.912219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1056/2130] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:28.963906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cfaf2-512b-42be-89f6-ae4d6ea3ab23 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:28.968898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:28.969776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:28.970061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:28.970334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] PATH_INFO: `/v3/users/8e97798fcb8e48369731981e75a2594c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:28.971932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e97798fcb8e48369731981e75a2594c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:28.973548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:29.210191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e97798fcb8e48369731981e75a2594c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.223882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e97798fcb8e48369731981e75a2594c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.329361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e97798fcb8e48369731981e75a2594c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.338371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e97798fcb8e48369731981e75a2594c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.339075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-707ba1c1-4f67-4873-8871-2deb120a004b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e97798fcb8e48369731981e75a2594c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.346294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1075/2131] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:28 2022] DELETE /identity/v3/users/8e97798fcb8e48369731981e75a2594c => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:29.469698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:29.471310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:29.484093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d758-695e-4e21-8f80-1d56ea4ca57d service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:29.552506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:29.553501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:29.553686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:29.553962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] PATH_INFO: `/v3/projects/0993ce09e1dd44a09c842964cae7948a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:29.557214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0993ce09e1dd44a09c842964cae7948a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:29.559238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:29.732443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0993ce09e1dd44a09c842964cae7948a', 'request_id': 'req-8042ca6f-7ddf-4cc3-b4af-356250a7022f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.835059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0993ce09e1dd44a09c842964cae7948a', 'request_id': 'req-8042ca6f-7ddf-4cc3-b4af-356250a7022f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.842990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0993ce09e1dd44a09c842964cae7948a', 'request_id': 'req-8042ca6f-7ddf-4cc3-b4af-356250a7022f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.854369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] The token cache is being invalidate because project 0993ce09e1dd44a09c842964cae7948a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:29.854951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.855627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8042ca6f-7ddf-4cc3-b4af-356250a7022f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:29.857385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1057/2132] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:29 2022] DELETE /identity/v3/projects/0993ce09e1dd44a09c842964cae7948a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:29.863847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:29.864334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:29.864434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:29.870335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:29.896643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:29.897857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:29.956223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1076/2133] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:29.964833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:29.965119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:29.965393 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:29.983989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:29.985115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:29.993566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1058/2134] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:29.997843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.005847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.007329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.009624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae50e05-3f2f-4ea1-ba70-36fd85e17fe1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.016768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.017445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.017737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.018437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.018780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:30.019098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.020724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.020860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d5ab7-72f6-4c25-8ed0-650efaa7ebd0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.028193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.029554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.029554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.029554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] PATH_INFO: `/v3/projects/c36db6691b6d4dd2bfee7559f6218381` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.030809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36db6691b6d4dd2bfee7559f6218381)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.032717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.034740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1077/2135] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:45:30.040488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.051987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308edd52-1d41-4197-814b-216ffde9d9ef admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.057185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.057909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.058196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.058657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.059310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.061638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.215557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1078/2136] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:30.219135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:30.221436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.233316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36db6691b6d4dd2bfee7559f6218381', 'request_id': 'req-737d31fa-d0ac-4962-b11d-605b82cd2213'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:30.292184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f8fe9-a977-4f76-bb0e-dc0723e25157 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.299371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.301371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.301371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.301371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.301977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.304291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.368269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36db6691b6d4dd2bfee7559f6218381', 'request_id': 'req-737d31fa-d0ac-4962-b11d-605b82cd2213'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:30.376084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36db6691b6d4dd2bfee7559f6218381', 'request_id': 'req-737d31fa-d0ac-4962-b11d-605b82cd2213'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:30.387593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] The token cache is being invalidate because project c36db6691b6d4dd2bfee7559f6218381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:30.387879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:30.388548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-737d31fa-d0ac-4962-b11d-605b82cd2213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:30.390210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1059/2137] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:30 2022] DELETE /identity/v3/projects/c36db6691b6d4dd2bfee7559f6218381 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:30.395152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24444f94-7ec1-40b1-9880-dce744b1a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.395440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24444f94-7ec1-40b1-9880-dce744b1a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.395723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24444f94-7ec1-40b1-9880-dce744b1a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.413840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-24444f94-7ec1-40b1-9880-dce744b1a1d2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:30.415625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24444f94-7ec1-40b1-9880-dce744b1a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.423636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1060/2138] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:30.431783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.432200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.432315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.452068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:30.453281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.459784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1079/2139] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:30.461614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1061/2140] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:30.465759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.466541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.468459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.480005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035812-0010-48e2-8b3c-193a6edfbc1d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.480615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d692d70-bcd3-4be8-b586-b3a741fedccf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.486443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.487676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.487892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.488212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.490763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:30.490763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.491614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.494274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.494274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.494767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.495088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.495855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.497474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.508215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1062/2141] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:30.516222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.517689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1080/2142] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:30.518808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.530347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c05385-5f43-42d9-9695-2881fb95cde3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.535061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.541544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.541544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.541544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.541544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.543564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efba8bf-ea2e-44cc-a277-6db32642bebc admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.543647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.548393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.549620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.549982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.550329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.551203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.553624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.568963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1081/2143] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:30.571975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.582770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413da17-39f7-4796-9236-14aff94bf4ff admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.587760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.588654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.588937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.589216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] PATH_INFO: `/v3/projects/e3e500b730cb4fb6a2841ff2750af31a/users/7ec7931fdf144b3ba5f16970e0e2b00a/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.607040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e500b730cb4fb6a2841ff2750af31a, user_id=7ec7931fdf144b3ba5f16970e0e2b00a, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.609698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b3285-c323-4710-8474-42a9b74c1850 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.664863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1063/2144] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/projects => generated 407 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:30.669166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.694894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1082/2145] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:30 2022] PUT /identity/v3/projects/e3e500b730cb4fb6a2841ff2750af31a/users/7ec7931fdf144b3ba5f16970e0e2b00a/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:30.701366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.701905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.702262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.728745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:30.732460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.765782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d9c68-e414-4306-9edf-89862fbad2ce admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.811670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1083/2146] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:30.821581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.823072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:30.842254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.843218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.843560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.843836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.844599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.846660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.886498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ed6ee-2bcd-43aa-9d88-7cab3fa24e0e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.890204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.890873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.891153 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.891424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.901779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.903283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.913735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1084/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:30.962299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1064/2148] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:30 2022] POST /identity/v3/users => generated 304 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:30.966501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:30.976214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb14a4a-b4fa-4d28-9386-6829554c74e1 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:30.982072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:30.982891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:30.983172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:30.983444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:30.984158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:30.985764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:30.996296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1085/2149] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:31.000341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.010825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5e6a2-de6d-45f2-886b-5218370cf34b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.015528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.016188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.016462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.016733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.017450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.019074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac37964-bce2-413e-921d-2e94aa8edf30 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.031690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1065/2150] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:31.035857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.047095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14821830-b480-41f1-8914-49b3b3de4c7a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.051751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.052645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.052928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.053205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] PATH_INFO: `/v3/projects/99db850fd3c44b1c9e6a8cd9ef168e06/users/ecf76d9412f24550b0c3501513a5c127/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.069738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99db850fd3c44b1c9e6a8cd9ef168e06, user_id=ecf76d9412f24550b0c3501513a5c127, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.073365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.175688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1086/2151] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:31 2022] PUT /identity/v3/projects/99db850fd3c44b1c9e6a8cd9ef168e06/users/ecf76d9412f24550b0c3501513a5c127/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:31.187626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.187913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.188187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.220651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] MFA Rules not processed for user `ecf76d9412f24550b0c3501513a5c127`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:31.225421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:31.295411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1066/2152] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:31.300091 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.370060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:31.380406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e132-a7d8-4ddd-9f2a-2c4d4725c56c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.385904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.386618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.386902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.387175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.387903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.389921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.533702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1087/2153] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:31 2022] POST /identity/v3/projects => generated 409 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:31.538013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.548839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc407bb-63a2-4328-8940-bb987c804f3f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.583210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:31.620135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.621439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.621439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.621646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.622345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.624401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.770188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1067/2154] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:31 2022] POST /identity/v3/users => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:31.774303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.784865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b380e-6ca0-423d-9dc5-9be93ed3b054 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.790103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.790803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.791140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.791426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.792154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.793793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.804489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1088/2155] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:31.809368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.820267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369b52b-205e-4fe1-89c9-98fd41858f86 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:31.825312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:31.826189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:31.826467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:31.826772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] PATH_INFO: `/v3/projects/a18803dc5d374245a2b50e199b04b317/users/5f1b79887ac74c9a991606404b40da96/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:31.842676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18803dc5d374245a2b50e199b04b317, user_id=5f1b79887ac74c9a991606404b40da96, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:31.845773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:31.939313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1068/2156] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:31 2022] PUT /identity/v3/projects/a18803dc5d374245a2b50e199b04b317/users/5f1b79887ac74c9a991606404b40da96/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:31.943326 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:31.953943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e9e51-e944-46b5-9348-76492f89a858 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:32.014795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:32.015613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:32.015906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:32.016182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:32.016916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:32.018561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4762a6-bbd4-49f5-a790-cd2f18e5a806 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:32.029638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1089/2157] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:32.034201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:32.046293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636ea76-2e57-4fbc-801d-4dc7a812a14d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:32.051398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:32.052313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:32.052591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:32.052860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] PATH_INFO: `/v3/projects/a18803dc5d374245a2b50e199b04b317/users/5f1b79887ac74c9a991606404b40da96/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:32.055750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18803dc5d374245a2b50e199b04b317, user_id=5f1b79887ac74c9a991606404b40da96, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:32.059088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:32.070304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1069/2158] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:32 2022] PUT /identity/v3/projects/a18803dc5d374245a2b50e199b04b317/users/5f1b79887ac74c9a991606404b40da96/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:32.081070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:32.081439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:32.081900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:32.112225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] MFA Rules not processed for user `5f1b79887ac74c9a991606404b40da96`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:32.113386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:32.186832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1090/2159] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:32.195323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:32.197547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:32.199279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:32.258997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db200-b0af-48a5-9605-04fc661c390b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:32.262849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:32.263498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:32.263776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:32.264046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:32.273106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:32.275216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:32.284662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1070/2160] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:32.721765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:32.723310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:32.739270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd07bda-9285-42e1-98e8-3737dccbc5d3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:32.740725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:32.741478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:32.741852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:32.742255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:32.813737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:32.815221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9fb428-9446-4985-b283-60e28f657656 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:32.830060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1091/2161] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:32.832656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:33.949550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:33.959412 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:34.091119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace6648-0975-4d9a-a8c0-4f634f2768b8 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:34.096564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:34.100101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:34.100101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:34.100101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] PATH_INFO: `/v3/users/0afdd60ae5a1444095657f5ce9d23fe5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:34.100397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afdd60ae5a1444095657f5ce9d23fe5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:34.102640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:34.396954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afdd60ae5a1444095657f5ce9d23fe5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:34.415540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afdd60ae5a1444095657f5ce9d23fe5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:34.480934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afdd60ae5a1444095657f5ce9d23fe5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:34.490626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afdd60ae5a1444095657f5ce9d23fe5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:34.491507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afdd60ae5a1444095657f5ce9d23fe5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:34.501852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1071/2162] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:33 2022] DELETE /identity/v3/users/0afdd60ae5a1444095657f5ce9d23fe5 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:34.503281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:34.504051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:34.504463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:34.517264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:34.525304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:34.526964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:34.597545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1092/2163] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:34.610267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:34.612248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:34.695129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8417-72f2-49bf-b5bc-eee10753de41 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:34.700516 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:34.701321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:34.701710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:34.702095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:34.714105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:34.715605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:34.730597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1072/2164] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:35.312953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:35.315446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:35.339019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ad4bb-93fb-47b4-9528-dec5f2161a80 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:35.367032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:35.368775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:35.368775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:35.370747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] PATH_INFO: `/v3/projects/109ba3059eb247d99e41eaf8b09cb5a7` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:35.371949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=109ba3059eb247d99e41eaf8b09cb5a7)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:35.374128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:35.515095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109ba3059eb247d99e41eaf8b09cb5a7', 'request_id': 'req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:35.525518 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109ba3059eb247d99e41eaf8b09cb5a7', 'request_id': 'req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:35.648379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109ba3059eb247d99e41eaf8b09cb5a7', 'request_id': 'req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:35.667635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] The token cache is being invalidate because project 109ba3059eb247d99e41eaf8b09cb5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:35.668035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:35.668794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:35.670663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1093/2165] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:35 2022] DELETE /identity/v3/projects/109ba3059eb247d99e41eaf8b09cb5a7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:35.675028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:35.689151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c1579-0e89-453c-8de7-8a511d16d02f service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:35.693601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:35.775010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:35.776396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:35.776875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:35.777178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] PATH_INFO: `/v3/users/9a54d638f60a4084a0e0925d8f93bb15` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:35.778722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a54d638f60a4084a0e0925d8f93bb15)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:35.781112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:36.110952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a54d638f60a4084a0e0925d8f93bb15'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:36.125244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a54d638f60a4084a0e0925d8f93bb15'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:36.223345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a54d638f60a4084a0e0925d8f93bb15'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:36.233110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a54d638f60a4084a0e0925d8f93bb15'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:36.233820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8b81a6b-3560-4d61-b514-117bdf8af282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a54d638f60a4084a0e0925d8f93bb15'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:36.240131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1073/2166] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:35 2022] DELETE /identity/v3/users/9a54d638f60a4084a0e0925d8f93bb15 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:37.085292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.087532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:37.120129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763baefe-11dc-4ca2-8a3d-3c8c4324dcc8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.220654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.222548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.223015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.223445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] PATH_INFO: `/v3/projects/d69b1345669e410c8d6922787e46110a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.225150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69b1345669e410c8d6922787e46110a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.227755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.337288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:37.372757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69b1345669e410c8d6922787e46110a', 'request_id': 'req-aa03d734-0d58-4eec-9f15-8a19798d1ccd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:37.380592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69b1345669e410c8d6922787e46110a', 'request_id': 'req-aa03d734-0d58-4eec-9f15-8a19798d1ccd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:37.461809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe6eb-2339-4150-b070-0f9c278c6097 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.462742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe6eb-2339-4150-b070-0f9c278c6097 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.463171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe6eb-2339-4150-b070-0f9c278c6097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.466278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69b1345669e410c8d6922787e46110a', 'request_id': 'req-aa03d734-0d58-4eec-9f15-8a19798d1ccd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:37.490679 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] The token cache is being invalidate because project d69b1345669e410c8d6922787e46110a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:37.491037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:37.492417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aa03d734-0d58-4eec-9f15-8a19798d1ccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:37.497389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1094/2167] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:37 2022] DELETE /identity/v3/projects/d69b1345669e410c8d6922787e46110a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:37.502875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.503256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.503598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.529357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:37.533610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:37.536505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-79ffe6eb-2339-4150-b070-0f9c278c6097 None None] MFA Rules not processed for user `7ec7931fdf144b3ba5f16970e0e2b00a`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:37.537742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79ffe6eb-2339-4150-b070-0f9c278c6097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:37.622158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1095/2168] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:37.623441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1074/2169] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:37.626991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.628615 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:37.630973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.631238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.631555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.646304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965dfca2-1c5f-444b-863f-0379fe7d92bb None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.652424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.653147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.653452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.653726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.654416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.655282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:37.656402 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.656531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:37.673415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1075/2170] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:37.674059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.683985 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2b819-7e23-44d0-995f-d1190978e4d5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.688791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.689475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.689746 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.690168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.690990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:37.691301 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.692929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.696010 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:37.707381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1076/2171] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:45:37.712268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.722571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6e9e-e9e6-454b-b1d5-a6335e8ab322 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.728413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.729123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.729409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.729690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.730390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.732699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.742844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1096/2172] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:37.744070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.816441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a63b3-d6cf-45f8-a61d-409e6d0b7307 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.822107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.823018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.823944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.824229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.824934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.826908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.831513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1077/2173] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:37.837148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.902649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d985d-51cc-48a9-911f-c4c44073c0fb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.907517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.908196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.908477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.908750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.909477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.913156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.922862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1097/2174] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:37.925104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.950508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1262f-49cf-4143-8991-11f495d169c3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:37.961415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:37.963384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:37.963610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:37.963890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:37.965154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:37.966878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:37.978869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1098/2175] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:37.983036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:37.996016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158fed23-fb18-44b3-849a-387395cb49c1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.002784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.003689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.003970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.004242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] PATH_INFO: `/v3/projects/b82620357804469d824314f3de90901b/users/141acad0eb384b1a81ed815dd25761a0/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.016480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1078/2176] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:37 2022] POST /identity/v3/users => generated 282 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:38.024193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.025170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82620357804469d824314f3de90901b, user_id=141acad0eb384b1a81ed815dd25761a0, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.028297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.035550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64301a00-0705-429e-b3d7-6929b10efd1f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.041741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.042477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.042796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.043068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.043797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.045566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.057262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1079/2177] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:38.061364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.076680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f942a4-c9af-4746-ba9d-0ce1bc434d3a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.083754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.084608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.084898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.085177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.086767 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.096446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.105482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1080/2178] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:38.106421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.112589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1099/2179] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:37 2022] PUT /identity/v3/projects/b82620357804469d824314f3de90901b/users/141acad0eb384b1a81ed815dd25761a0/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:38.120269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7ec50-1f5f-4f06-9cfd-d61b080836f9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.120569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.174174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:38.200463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.201644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.201932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.202208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] PATH_INFO: `/v3/projects/038e60adffd442c8acc6735bc8b540bd/users/3f5bdc6dce87468b8b84c8796fc1cc3e/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.204042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c15dd-f2b1-44e3-8c86-f40c9a61727f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.209271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.210129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.210787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.211225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.212010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.219183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.219352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=038e60adffd442c8acc6735bc8b540bd, user_id=3f5bdc6dce87468b8b84c8796fc1cc3e, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.221908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.225517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1100/2180] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:38.229880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.241448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d176e-3844-45e4-86f6-ae528b9150aa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.247262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.248173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.248453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.248729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] PATH_INFO: `/v3/projects/b82620357804469d824314f3de90901b/users/141acad0eb384b1a81ed815dd25761a0/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.250841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b82620357804469d824314f3de90901b, user_id=141acad0eb384b1a81ed815dd25761a0, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.253865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682fa504-621e-4b07-986e-83b519b6ed2c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.265750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1101/2181] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:38 2022] PUT /identity/v3/projects/b82620357804469d824314f3de90901b/users/141acad0eb384b1a81ed815dd25761a0/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:38.328456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1081/2182] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:38 2022] PUT /identity/v3/projects/038e60adffd442c8acc6735bc8b540bd/users/3f5bdc6dce87468b8b84c8796fc1cc3e/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:38.339945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.340264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.340511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.384048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] MFA Rules not processed for user `3f5bdc6dce87468b8b84c8796fc1cc3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:38.385299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:38.475633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1102/2183] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:38.476542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.562504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f310-63a1-4e9d-b6f3-af3115346594 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.566569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.567241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.567519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.567793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.568770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.571290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.739751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:38.747718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1082/2184] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:38 2022] POST /identity/v3/projects => generated 363 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:38.751751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.763012 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17680822-7078-45ef-bc69-8d20ca54c0b4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.820178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.820848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.821173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.821438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.822121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.824066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.922779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1103/2185] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:38 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:38.926502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.939667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194def2-0a30-4cf5-b3d4-1c27226ad279 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.944801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.945466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.945954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.946241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:38.947061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:38.948701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:38.965917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1083/2186] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:38.969991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:38.985019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0f3a-0156-4476-8f96-294d3ff42f20 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:38.990754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:38.991986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:38.992617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:38.995061 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] PATH_INFO: `/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170/users/98ec8ef880de4eeb9c00d04328f08d72/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.012669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906d3e6cd7b34f33b7f471ba4a40d170, user_id=98ec8ef880de4eeb9c00d04328f08d72, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:39.015517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:39.121011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:39.137990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1104/2187] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:38 2022] PUT /identity/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170/users/98ec8ef880de4eeb9c00d04328f08d72/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:39.141843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:39.161745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff1f88-15ef-47db-9a38-21ff9ccf38ee admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:39.224694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:39.225523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:39.225812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:39.226109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.227602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:39.230598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb125e7d-3498-4e9c-8832-46baa93cb363 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:39.252628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1084/2188] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:39.252814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:39.259691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2f2eb-a9a8-462d-a260-048500337b26 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:39.264605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:39.265500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:39.265781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:39.266055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] PATH_INFO: `/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170/users/98ec8ef880de4eeb9c00d04328f08d72/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.268128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906d3e6cd7b34f33b7f471ba4a40d170, user_id=98ec8ef880de4eeb9c00d04328f08d72, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:39.270943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:39.283752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1105/2189] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:39 2022] PUT /identity/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170/users/98ec8ef880de4eeb9c00d04328f08d72/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:39.295165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:39.295389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:39.295665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.325341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] MFA Rules not processed for user `98ec8ef880de4eeb9c00d04328f08d72`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:39.326756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:39.409638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1085/2190] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:39.421782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:39.423546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:39.490687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee715e-e0e2-4883-95eb-5e74bf7294d9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:39.495053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:39.495800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:39.496158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:39.496507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.509077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:39.510622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eed3c3-af50-4c68-ad09-ebb62af9b253 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:39.521997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1106/2191] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:39 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:39.775804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:39.787648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc26a93-0a78-404b-b84a-8d9ef659b4fe None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:39.792450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:39.793300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:39.793979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:39.794422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:39.826274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:39.896045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:39.896045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:39.906660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1086/2192] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:39 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:45.419282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:45.419823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:45.419823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:45.450176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] MFA Rules not processed for user `141acad0eb384b1a81ed815dd25761a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:45.451933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:45.530696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1107/2193] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:45.545415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:45.545415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:45.664233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c00a2-dd4e-4c4c-a022-d65b740c1b70 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:45.670611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:45.671377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:45.671727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:45.672007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:45.747252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:45.749032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c60e33-2031-4db9-b56f-539792f22596 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:45.760101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1087/2194] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:46.087352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:46.142216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:46.159090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fdff7-d4bf-4245-a3b6-54222a97d213 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:46.164235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:46.169336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:46.169336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:46.169336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] PATH_INFO: `/v3/users/3f5bdc6dce87468b8b84c8796fc1cc3e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:46.169336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5bdc6dce87468b8b84c8796fc1cc3e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:46.169336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:46.425277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5bdc6dce87468b8b84c8796fc1cc3e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:46.440946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5bdc6dce87468b8b84c8796fc1cc3e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:46.542888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5bdc6dce87468b8b84c8796fc1cc3e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:46.550816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5bdc6dce87468b8b84c8796fc1cc3e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:46.551536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5bdc6dce87468b8b84c8796fc1cc3e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:46.558348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1108/2195] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:46 2022] DELETE /identity/v3/users/3f5bdc6dce87468b8b84c8796fc1cc3e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:46.562771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:46.563294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:46.563415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:46.581912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:46.583086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:46.613788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:46.654802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1088/2196] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:46.675235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:46.676748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:46.750646 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad4436-c138-4b03-8cf2-70272ae20b34 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:46.754883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:46.755629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:46.756183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:46.756461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:46.767768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:46.769301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:46.781888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1109/2197] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:47.322287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:47.324300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:47.342276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774dd8-a42a-4fde-8078-21cabaab8663 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:47.347778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:47.348645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:47.348933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:47.349209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] PATH_INFO: `/v3/projects/038e60adffd442c8acc6735bc8b540bd` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:47.350414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038e60adffd442c8acc6735bc8b540bd)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:47.353004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:47.604375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038e60adffd442c8acc6735bc8b540bd', 'request_id': 'req-eec2d156-a20d-4433-bffe-953ef95cd0c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:47.786044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038e60adffd442c8acc6735bc8b540bd', 'request_id': 'req-eec2d156-a20d-4433-bffe-953ef95cd0c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:47.796705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038e60adffd442c8acc6735bc8b540bd', 'request_id': 'req-eec2d156-a20d-4433-bffe-953ef95cd0c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:47.811193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] The token cache is being invalidate because project 038e60adffd442c8acc6735bc8b540bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:47.811656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:47.817630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:47.817630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1089/2198] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:47 2022] DELETE /identity/v3/projects/038e60adffd442c8acc6735bc8b540bd => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:47.819773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:47.832534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf95f-e235-4ec5-902b-d11801fad670 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:47.912127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:47.913621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:47.913925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:47.914370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] PATH_INFO: `/v3/users/98ec8ef880de4eeb9c00d04328f08d72` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:47.915859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ec8ef880de4eeb9c00d04328f08d72)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:47.917814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:47.935443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:48.200616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ec8ef880de4eeb9c00d04328f08d72'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:48.221766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ec8ef880de4eeb9c00d04328f08d72'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:48.383726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ec8ef880de4eeb9c00d04328f08d72'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:48.396851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ec8ef880de4eeb9c00d04328f08d72'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:48.398112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0f78bc06-592c-470b-bdae-048fbba2fb7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ec8ef880de4eeb9c00d04328f08d72'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:48.409762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1110/2199] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:47 2022] DELETE /identity/v3/users/98ec8ef880de4eeb9c00d04328f08d72 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:49.068976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:49.070779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:49.085679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d156-a20d-4433-bffe-953ef95cd0c4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:49.203716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:49.205075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.205535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.205786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] PATH_INFO: `/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.208478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=906d3e6cd7b34f33b7f471ba4a40d170)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:49.211973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:49.338587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '906d3e6cd7b34f33b7f471ba4a40d170', 'request_id': 'req-872e67aa-aadd-490f-b360-4036e957e8d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:49.342400 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:49.443405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '906d3e6cd7b34f33b7f471ba4a40d170', 'request_id': 'req-872e67aa-aadd-490f-b360-4036e957e8d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:49.458198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '906d3e6cd7b34f33b7f471ba4a40d170', 'request_id': 'req-872e67aa-aadd-490f-b360-4036e957e8d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:49.474224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] The token cache is being invalidate because project 906d3e6cd7b34f33b7f471ba4a40d170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:49.474570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:49.476048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-872e67aa-aadd-490f-b360-4036e957e8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:49.478295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1090/2200] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:49 2022] DELETE /identity/v3/projects/906d3e6cd7b34f33b7f471ba4a40d170 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:49.483307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.483781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.483871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.503514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:49.505062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:49.586427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1111/2201] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:49.595425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.595706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.596162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.620221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:49.621435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:49.629744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1091/2202] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:49.634815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:49.644673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958e664-e5ff-4e00-b388-7debbdec2c20 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:49.650138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:49.650963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.651336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.651701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.652559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:49.652964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:49.654799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:49.671102 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1112/2203] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:45:49.672542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:49.684581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493de32-2711-4487-b3e6-53983c1ed981 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:49.689737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:49.690472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.690876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.691230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.692052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:49.695637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:49.745515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:49.747491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:49.762951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd097e-b24b-41ee-832b-b53628da436a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:49.770643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:49.771538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.771826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.772107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] PATH_INFO: `/v3/users/72bebe9eec964729bc436d023dbdd288` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.774365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72bebe9eec964729bc436d023dbdd288)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:49.776637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:49.904908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1092/2204] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:49 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:49.905810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:49.972447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c4d3f-bfd6-4de7-b6ee-a6de8f209978 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:49.978141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:49.978835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:49.979115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:49.979390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:49.980081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:49.982082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.079643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72bebe9eec964729bc436d023dbdd288'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:50.098099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72bebe9eec964729bc436d023dbdd288'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:50.126996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1093/2205] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:49 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:50.128091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.138694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2ef2-141f-4726-85fc-6b66a7af2f28 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.143374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.144231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.144364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.144629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.145391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.145695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:50.147384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.157909 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1094/2206] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:50.161606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.171542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb24caa-6c60-4314-8247-0dd40d321e17 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.176562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.177199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.177484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.177872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.179205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.180948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.191741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1095/2207] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:50.196422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.205889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac0a90f-e63e-4a46-93ae-c788c9bbe645 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.207980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72bebe9eec964729bc436d023dbdd288'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:50.218328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.218328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.218328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.218328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] PATH_INFO: `/v3/projects/7c729ea5959e4581957ca5186718c804/users/839c7803f79c4bdf97522a0b66e2fb56/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.218851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72bebe9eec964729bc436d023dbdd288'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:50.218851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-ae216d30-a659-4d73-839e-f4c593176f0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72bebe9eec964729bc436d023dbdd288'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:50.224843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1113/2208] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:49 2022] DELETE /identity/v3/users/72bebe9eec964729bc436d023dbdd288 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:50.230075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.230484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.230829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.231593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c729ea5959e4581957ca5186718c804, user_id=839c7803f79c4bdf97522a0b66e2fb56, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.234266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edb861-28d3-4e22-8c46-efdc427fa598 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.249916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:50.251169 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:50.315231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1114/2209] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:50.325440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.328715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:50.345872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1096/2210] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:50 2022] PUT /identity/v3/projects/7c729ea5959e4581957ca5186718c804/users/839c7803f79c4bdf97522a0b66e2fb56/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:50.360628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.361805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.362107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.401543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] MFA Rules not processed for user `839c7803f79c4bdf97522a0b66e2fb56`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:50.402923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:50.423814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1736-dfa4-4ebb-b7e2-f170604e1d02 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.484756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1097/2211] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:50.489819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.492581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:50.500917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.501581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.501914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.502237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.568085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac1950-1338-4f33-a824-3e8655e8f88c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.574468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.575483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.575605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.575877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.576009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.576614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.578928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b20c1-6680-4d3a-bfa2-5e04643d84e7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.581010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.595066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1115/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:50.753321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1098/2213] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:50 2022] POST /identity/v3/projects => generated 379 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:50.758623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.759388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.759835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.760185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.768761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce077a60-1546-48a5-8db5-fd123acfbb1a service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:50.778224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] MFA Rules not processed for user `174b3629771b4044bb7b1cee523225d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:50.779470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:50.853752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:50.854799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:50.855277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:50.855563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:50.856355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:50.858698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:50.863544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1099/2214] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:50.976409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1116/2215] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:50 2022] POST /identity/v3/users => generated 290 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:50.977924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:50.995992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bc8a6-10b7-4a77-8706-870592c372f8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.002076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.002852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.005915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.005915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.005915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.005915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.016591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1100/2216] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:51.020769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.031890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5edd2-53d1-4550-bbed-05c0a46c6737 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.036587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.037258 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.037541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.038682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.039988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.040799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.051543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1117/2217] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:51.056250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.066723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3546-e09c-447a-b6e5-2f0635dd73c1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.071557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.072547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.073857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.074229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] PATH_INFO: `/v3/projects/50c14cf51638435a9759eee6fc673623/users/74bb46a1f56c4026ab5f7702f8053f0b/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.089754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:51.093146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c14cf51638435a9759eee6fc673623, user_id=74bb46a1f56c4026ab5f7702f8053f0b, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.096071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0911c6b-9214-47d6-a35a-84e622955a7e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.139552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.141939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.159968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700b692-d003-4ae3-aa40-fe393fff24eb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.167147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.171556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.171556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.171556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] PATH_INFO: `/v3/projects/ee76ebc17e684086b0573655b1d4b6af` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.171556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee76ebc17e684086b0573655b1d4b6af)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.173004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.255154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1101/2218] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:51 2022] PUT /identity/v3/projects/50c14cf51638435a9759eee6fc673623/users/74bb46a1f56c4026ab5f7702f8053f0b/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:51.266015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.266420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.266737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.297941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] MFA Rules not processed for user `74bb46a1f56c4026ab5f7702f8053f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:51.299570 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.331554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee76ebc17e684086b0573655b1d4b6af', 'request_id': 'req-859a5741-31dc-4652-b379-713dd081af1e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:51.339770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee76ebc17e684086b0573655b1d4b6af', 'request_id': 'req-859a5741-31dc-4652-b379-713dd081af1e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:51.377082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1102/2219] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:51.382337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.435297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee76ebc17e684086b0573655b1d4b6af', 'request_id': 'req-859a5741-31dc-4652-b379-713dd081af1e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:51.458946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b3b6d-3c45-4d70-9efc-5f3f21b7b1eb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.469069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] The token cache is being invalidate because project ee76ebc17e684086b0573655b1d4b6af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:51.469591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:51.470889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-859a5741-31dc-4652-b379-713dd081af1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:51.473692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1118/2220] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:51 2022] DELETE /identity/v3/projects/ee76ebc17e684086b0573655b1d4b6af => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:51.479470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d201-e7cb-4df3-8b1b-2b5ee7f0ee89 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.479907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d201-e7cb-4df3-8b1b-2b5ee7f0ee89 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.480536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5d201-e7cb-4df3-8b1b-2b5ee7f0ee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.503570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2dc5d201-e7cb-4df3-8b1b-2b5ee7f0ee89 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:51.504781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2dc5d201-e7cb-4df3-8b1b-2b5ee7f0ee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.534252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.535046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.535394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.535679 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.536382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.539320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.585859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1119/2221] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:51.594487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.594808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.595148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.618499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:51.619775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.627828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1120/2222] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:51.632297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.641577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5633ed1-0332-4b37-8bad-046991e2e7fe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.646418 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.647297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.647933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.647933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.648704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:51.652436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.652436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.664929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1121/2223] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 14:45:51.669357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.680517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48714e-a4b0-4070-af72-0391858f6bf6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.685575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.689427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.689427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.689427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.689427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.689769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.704614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1103/2224] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/projects => generated 379 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:51.705699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.707624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.785031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a991a3-6041-46b5-95e0-35fae91ea9a1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.791645 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.792471 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.792771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.793110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.793869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.796025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.818183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1122/2225] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:51.818183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.818183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:51.892466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250c8eb-034b-4207-afc7-16c4f62e002e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.898158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.899291 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.902974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.903123 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1104/2226] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:51.903210 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] PATH_INFO: `/v3/users/ecf76d9412f24550b0c3501513a5c127` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.903287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:51.914434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf76d9412f24550b0c3501513a5c127)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.914434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:51.915541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e943d-0f24-4197-8c53-2bcf6004d63f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:51.920963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:51.921641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:51.921922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:51.922243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:51.922991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:51.924926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.071915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1105/2227] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:51 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:52.071915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.080910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48bba7-7068-4338-9495-d9a87e700132 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.087095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.087859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.088212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.088559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.089367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.091073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.101820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1106/2228] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:52.102858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.112800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13528cd6-4ebe-4931-918b-1c65d39bc767 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.117564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.118211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.118490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.118797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.119519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.121132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.132900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1107/2229] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:52.133521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.143944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133766c-a767-4a95-a6c4-b2fe7fc3ab65 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.149531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.150356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.150671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.150948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] PATH_INFO: `/v3/projects/210150dcc63f40d6925991e2fdac9afb/users/80655d49f93d4147b8b5006afb2a33c5/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.166209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210150dcc63f40d6925991e2fdac9afb, user_id=80655d49f93d4147b8b5006afb2a33c5, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.168988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.215886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf76d9412f24550b0c3501513a5c127'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:52.232087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf76d9412f24550b0c3501513a5c127'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:52.239719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:52.325600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1108/2230] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:52 2022] PUT /identity/v3/projects/210150dcc63f40d6925991e2fdac9afb/users/80655d49f93d4147b8b5006afb2a33c5/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:52.326998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.344867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf76d9412f24550b0c3501513a5c127'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:52.359791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf76d9412f24550b0c3501513a5c127'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:52.365637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf76d9412f24550b0c3501513a5c127'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:52.376895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1123/2231] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:51 2022] DELETE /identity/v3/users/ecf76d9412f24550b0c3501513a5c127 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:52.378116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.388548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.403225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ec039-a7f8-452a-90d9-6f5407e1d848 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.456710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c8bc-6dfe-4b79-aa86-5971f60625dd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.463375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.464086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.464355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.464657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.465361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.466451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.467101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbb82e6-202e-40ea-80d9-d1183eb617b8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.467221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.467427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.467699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.468464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.470075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.481624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1124/2232] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:52.481624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.481624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.481624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.482117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1109/2233] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:52.482589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.493856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0331def-cad1-47e7-9648-c792caffb8e2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.498775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.499658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.499934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.500205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] PATH_INFO: `/v3/projects/210150dcc63f40d6925991e2fdac9afb/users/80655d49f93d4147b8b5006afb2a33c5/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.501577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:52.502283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210150dcc63f40d6925991e2fdac9afb, user_id=80655d49f93d4147b8b5006afb2a33c5, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.503062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.505641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.518133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1125/2234] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:52.518133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.518133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.523032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1110/2235] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:52 2022] PUT /identity/v3/projects/210150dcc63f40d6925991e2fdac9afb/users/80655d49f93d4147b8b5006afb2a33c5/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:52.525895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.527993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.532013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dddbff-886b-4258-a709-1f468bdec056 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.593422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57703c3a-d67d-484f-ba56-d2107256a5f7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.597259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.597882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.598162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.598437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.613885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.615216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.615689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.617510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] PATH_INFO: `/v3/projects/5bd07538140e49dc80b0d61a1a404754/users/218c2788d21f4912822841185afbfaf0/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.634654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd07538140e49dc80b0d61a1a404754, user_id=218c2788d21f4912822841185afbfaf0, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.643384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2784232-be2c-42a6-9c17-d45f5fe5423f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.667143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.668589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a7a55-d86f-4b3d-8446-3541c9b31da5 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.679022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1111/2236] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:52.681402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.681896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.682387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.715600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] MFA Rules not processed for user `80655d49f93d4147b8b5006afb2a33c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:52.716885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.766833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1126/2237] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:52 2022] PUT /identity/v3/projects/5bd07538140e49dc80b0d61a1a404754/users/218c2788d21f4912822841185afbfaf0/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:52.772349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.772668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.773044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.790904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:52.792117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.799658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1112/2238] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:52.817332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.826847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:52.857780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1127/2239] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:52.872670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:52.916221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598c1a-ea1a-4d7b-8b6b-b96c1fc8bc2c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.937387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e50bd9-e6cc-4da9-9784-51ea8931ca4a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:52.943590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:52.944272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:52.944553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:52.944827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:52.957072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:52.958485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:52.973124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1128/2240] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:53.001409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:53.002848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:53.003534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:53.003910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:53.071459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:53.073076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:53.084760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1113/2241] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:53.204511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:53.206013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:53.219314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69efce-4891-44f1-9f1b-6d36f04aa4b5 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:53.222963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:53.223644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:53.223909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:53.224173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:53.226774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:53.283312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:53.292765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:53.294365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:53.300492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:53.301959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:53.319840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5584c6-8d3a-4398-b0bc-407ecf3b2154 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:53.331068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:53.332854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:53.333386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:53.333681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] PATH_INFO: `/v3/projects/99db850fd3c44b1c9e6a8cd9ef168e06` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:53.334410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1129/2242] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:53.335291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99db850fd3c44b1c9e6a8cd9ef168e06)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:53.337371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:53.521091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99db850fd3c44b1c9e6a8cd9ef168e06', 'request_id': 'req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:53.700005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99db850fd3c44b1c9e6a8cd9ef168e06', 'request_id': 'req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:53.707740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99db850fd3c44b1c9e6a8cd9ef168e06', 'request_id': 'req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:53.718855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] The token cache is being invalidate because project 99db850fd3c44b1c9e6a8cd9ef168e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:53.719153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:53.719855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:53.721547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1114/2243] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:53 2022] DELETE /identity/v3/projects/99db850fd3c44b1c9e6a8cd9ef168e06 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:53.726282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:53.739057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f80cdc-fb6d-461a-962d-21d290a8ed5d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:53.807416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:53.808293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:53.808580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:53.808948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] PATH_INFO: `/v3/users/5f1b79887ac74c9a991606404b40da96` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:53.810164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1b79887ac74c9a991606404b40da96)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:53.812176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:54.019453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:54.093875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1b79887ac74c9a991606404b40da96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:54.115508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1b79887ac74c9a991606404b40da96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:54.230792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1b79887ac74c9a991606404b40da96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:54.238622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1b79887ac74c9a991606404b40da96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:54.239242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-81044417-ba48-4ab7-bcea-f033687b2744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1b79887ac74c9a991606404b40da96'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:54.246027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1130/2244] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:53 2022] DELETE /identity/v3/users/5f1b79887ac74c9a991606404b40da96 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:54.764690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:54.771286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:54.787897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa58175-5cbf-4ef5-82a9-e5394926b1e3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:54.857684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:54.858597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:54.858881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:54.859151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] PATH_INFO: `/v3/projects/a18803dc5d374245a2b50e199b04b317` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:54.860343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18803dc5d374245a2b50e199b04b317)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:54.863079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:54.863535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:54.863859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:54.868491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:54.895247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] MFA Rules not processed for user `218c2788d21f4912822841185afbfaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:54.896656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:54.977955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1131/2245] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:54.989141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:54.991068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:55.037313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18803dc5d374245a2b50e199b04b317', 'request_id': 'req-e19119dd-4ded-4003-a0fb-95751ff97623'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:55.049714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86291e-77c4-4737-ac7d-55876d76558f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.099814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.101086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.101365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.101636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.139328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18803dc5d374245a2b50e199b04b317', 'request_id': 'req-e19119dd-4ded-4003-a0fb-95751ff97623'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:55.148109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18803dc5d374245a2b50e199b04b317', 'request_id': 'req-e19119dd-4ded-4003-a0fb-95751ff97623'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:55.160899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] The token cache is being invalidate because project a18803dc5d374245a2b50e199b04b317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:55.161178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:55.161829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e19119dd-4ded-4003-a0fb-95751ff97623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:55.164178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1115/2246] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:54 2022] DELETE /identity/v3/projects/a18803dc5d374245a2b50e199b04b317 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:55.164799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.166463 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec41116-3c3f-4538-83a7-d23d349ffcc9 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.168918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.169189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.169465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.176297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1132/2247] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:45:55.196949 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:55.198425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:55.266903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1116/2248] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:55.281168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.281495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.281756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.301476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:55.302721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:55.314949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1133/2249] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:55.320137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.331241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9e0c4-e2dc-48aa-87be-ab01fee44ad0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.336666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.337337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.337617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.337890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.338658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:45:55.338966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.340621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.363089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1117/2250] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:45:55.365238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.379705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e004f735-9228-4a51-a6c5-b950464c9672 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.385985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.386731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.387013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.387287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.387979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.390094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.505980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.506484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1134/2251] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:55 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:55.522459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db0694-d105-464b-b226-6b4d7ca2fd75 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.587537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.588357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.588649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.588931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.591017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.592993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.728798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1118/2252] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:55 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:55.732882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.743579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df5e06-e418-4285-9194-3e41fb5f7f25 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.748775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.753816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.753816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.753816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.753816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.753816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.768870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1135/2253] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:55.771048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.792176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62ec78-6392-4c05-9972-5ad0f1ba80ff admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.798423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.799132 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.799476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.799787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.800521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.803488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cce41-5ee1-4f39-95b7-7e69965ddb0e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.830942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1119/2254] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:55 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:45:55.831057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:55.840954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9747ada6-5d7c-4114-97fb-e98a9b56d3f0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:55.845955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:55.846882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.847166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.847439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] PATH_INFO: `/v3/projects/060d065435064c1094dcdabfa6260965/users/133cec7515bd41119c96f2f8713edd11/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:55.865843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060d065435064c1094dcdabfa6260965, user_id=133cec7515bd41119c96f2f8713edd11, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:55.868777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:55.919781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:55.963688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1136/2255] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:55 2022] PUT /identity/v3/projects/060d065435064c1094dcdabfa6260965/users/133cec7515bd41119c96f2f8713edd11/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:55.976799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:55.976799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:55.976799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.005157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] MFA Rules not processed for user `133cec7515bd41119c96f2f8713edd11`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:56.007231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:56.087435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1120/2256] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:56.092274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.167182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89154c21-5a2a-4202-b337-33f9aaf7e7e9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.172230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.172898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.173179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.173551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.174130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.176064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.284965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1137/2257] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:45:56 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:45:56.288388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.305406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d38ff8-eefa-4a2a-b798-14cfb897f682 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.378245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.378961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.379243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.379568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.380286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.383068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.439301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:56.518791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1121/2258] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:45:56 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:45:56.520244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.530709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89c86-8a09-40b2-8642-335e1c06ad91 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.535628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.536333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.536616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.536888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.538440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.540092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.554893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1138/2259] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:56.556212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.578992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3dd5c4-23e7-443a-b7f0-1c0808bc7921 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.586059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.587050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.587335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.587614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] PATH_INFO: `/v3/projects/4df1b32040574608ba8650a725b86c1b/users/12323fd621214c6e8d7ad5746df744d0/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.607344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df1b32040574608ba8650a725b86c1b, user_id=12323fd621214c6e8d7ad5746df744d0, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.610559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.713910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1122/2260] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:56 2022] PUT /identity/v3/projects/4df1b32040574608ba8650a725b86c1b/users/12323fd621214c6e8d7ad5746df744d0/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:56.715629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.727866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e3330-a44d-4d67-a9e7-d1c91acf925f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.796134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.796805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.797084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.797356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.798193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.805107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807c27-1f09-40c3-a62b-206d3b08eec1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.810854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1139/2261] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:45:56.814928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:56.826022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade00cb-879d-4406-b93a-66afcd60356e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:56.832697 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:56.833604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.833882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.834703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] PATH_INFO: `/v3/projects/4df1b32040574608ba8650a725b86c1b/users/12323fd621214c6e8d7ad5746df744d0/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.837013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df1b32040574608ba8650a725b86c1b, user_id=12323fd621214c6e8d7ad5746df744d0, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:56.840048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:56.859473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1123/2262] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:45:56 2022] PUT /identity/v3/projects/4df1b32040574608ba8650a725b86c1b/users/12323fd621214c6e8d7ad5746df744d0/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:56.874410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:56.874410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:56.874410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:56.903437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] MFA Rules not processed for user `12323fd621214c6e8d7ad5746df744d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:56.904836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:56.996022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1140/2263] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:57.018233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:57.018233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:57.093217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96b765-11ac-46b0-a0d3-4d194c36da1d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:57.097030 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:57.099033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:57.099329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:57.099615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:57.112682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:57.128433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1ccaa4-77ee-45bf-ae79-70751b6d7073 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:57.128433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1124/2264] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:58.160236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:58.225485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25949859-6de4-43ef-afd8-471e839eafdd None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:58.230099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:58.230995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:58.231278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:58.231552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] PATH_INFO: `/v3/users/133cec7515bd41119c96f2f8713edd11` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:58.232735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=133cec7515bd41119c96f2f8713edd11)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:58.234689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:58.507610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133cec7515bd41119c96f2f8713edd11'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:58.519145 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133cec7515bd41119c96f2f8713edd11'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:58.619205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133cec7515bd41119c96f2f8713edd11'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:58.626769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133cec7515bd41119c96f2f8713edd11'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:58.629132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-2fce1e0e-1cb0-4286-a4f6-df15428fcfcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133cec7515bd41119c96f2f8713edd11'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:58.634098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1141/2265] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:58 2022] DELETE /identity/v3/users/133cec7515bd41119c96f2f8713edd11 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:45:58.639293 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:58.639814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:58.640091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:58.658807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:58.663678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:58.708664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:58.708916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:58.709691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:58.728302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] MFA Rules not processed for user `218c2788d21f4912822841185afbfaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:45:58.729586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:58.733424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1125/2266] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:45:58.742410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:58.743942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:58.790312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1142/2267] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:45:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:45:58.802462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b232d81-d053-42ba-8cd4-c7839adee164 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:58.814216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:58.814216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:58.814216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:58.814216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:58.819512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:58.821466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:58.832078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1126/2268] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:45:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:45:58.900060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:59.307888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:59.310209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:59.327860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606edd35-c02e-45cb-9f68-7c25f2a2b7a7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:59.334790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:59.336140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:59.336754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:59.337033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] PATH_INFO: `/v3/projects/060d065435064c1094dcdabfa6260965` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:59.338627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060d065435064c1094dcdabfa6260965)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:59.340923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:59.495243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060d065435064c1094dcdabfa6260965', 'request_id': 'req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.501493 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060d065435064c1094dcdabfa6260965', 'request_id': 'req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.639191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060d065435064c1094dcdabfa6260965', 'request_id': 'req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.655353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] The token cache is being invalidate because project 060d065435064c1094dcdabfa6260965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:45:59.655640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.656491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.658660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1143/2269] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:45:59 2022] DELETE /identity/v3/projects/060d065435064c1094dcdabfa6260965 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:45:59.664335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:59.678870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68072e-d7dd-42a7-bbf2-fbc95b1a716e service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:59.761852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:59.762765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:59.763042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:59.763315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] PATH_INFO: `/v3/users/12323fd621214c6e8d7ad5746df744d0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:59.764512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12323fd621214c6e8d7ad5746df744d0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:59.767337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:59.819855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:45:59.821379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:45:59.836227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43a566-a3ee-4c54-a07e-a1fcb174f5a4 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:45:59.842037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:45:59.848952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:45:59.848952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:45:59.848952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] PATH_INFO: `/v3/users/839c7803f79c4bdf97522a0b66e2fb56` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:45:59.848952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=839c7803f79c4bdf97522a0b66e2fb56)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:45:59.848952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:45:59.960547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12323fd621214c6e8d7ad5746df744d0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:45:59.962902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:45:59.974276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12323fd621214c6e8d7ad5746df744d0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.085408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12323fd621214c6e8d7ad5746df744d0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.093859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12323fd621214c6e8d7ad5746df744d0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.095454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e31c8cc-b361-4596-8216-b5952552fae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12323fd621214c6e8d7ad5746df744d0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.103623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839c7803f79c4bdf97522a0b66e2fb56'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.105691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1127/2270] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:59 2022] DELETE /identity/v3/users/12323fd621214c6e8d7ad5746df744d0 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:00.120608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839c7803f79c4bdf97522a0b66e2fb56'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.187798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839c7803f79c4bdf97522a0b66e2fb56'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.195964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839c7803f79c4bdf97522a0b66e2fb56'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.196667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839c7803f79c4bdf97522a0b66e2fb56'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.202876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1144/2271] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:45:59 2022] DELETE /identity/v3/users/839c7803f79c4bdf97522a0b66e2fb56 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:00.208054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:00.208344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:00.208668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:00.229984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:00.232473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:00.233440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:00.307283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1128/2272] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:46:00.317451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:00.319521 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:00.382953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad01d6-860c-403c-aaf0-1f824aba5dfb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:00.386829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:00.387479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:00.387756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:00.388024 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:00.405315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:00.407338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:00.418158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1145/2273] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:00.620220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:00.621894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:00.635859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd375b-342d-45b9-a20c-be60e2d57c53 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:00.641108 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:00.642032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:00.642561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:00.642696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] PATH_INFO: `/v3/projects/4df1b32040574608ba8650a725b86c1b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:00.644237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df1b32040574608ba8650a725b86c1b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:00.646306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:00.798987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df1b32040574608ba8650a725b86c1b', 'request_id': 'req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.884523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df1b32040574608ba8650a725b86c1b', 'request_id': 'req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.887007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:00.893266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df1b32040574608ba8650a725b86c1b', 'request_id': 'req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.901553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:00.903221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:00.905175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] The token cache is being invalidate because project 4df1b32040574608ba8650a725b86c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:00.905531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.906127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14804dcb-cdc1-4ce5-a3f2-ced35fcfba1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:00.907942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1129/2274] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:00 2022] DELETE /identity/v3/projects/4df1b32040574608ba8650a725b86c1b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:00.913218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a8f0e-feca-4b09-8762-3ff8a2632ed7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:00.913637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a8f0e-feca-4b09-8762-3ff8a2632ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:00.913922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a8f0e-feca-4b09-8762-3ff8a2632ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:00.917755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66fdc7-5348-4016-90c4-68296360e505 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:00.949651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-994a8f0e-feca-4b09-8762-3ff8a2632ed7 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:00.950953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-994a8f0e-feca-4b09-8762-3ff8a2632ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:00.989234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:00.990644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:00.991228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:00.991501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] PATH_INFO: `/v3/projects/7c729ea5959e4581957ca5186718c804` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:00.992974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c729ea5959e4581957ca5186718c804)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:00.995006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.020300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1130/2275] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:01.028920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.029221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.029498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.049448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:01.050683 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:01.065665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1131/2276] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:46:01.067441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.080799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba9a36-79e9-4fab-86fc-8a6095d89375 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.086681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.087367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.087648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.087924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.088802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:01.089118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.090783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.108096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1132/2277] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:01.109129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.119776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadd564-ebfa-4563-9696-74d28422a894 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.127566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c729ea5959e4581957ca5186718c804', 'request_id': 'req-cd994011-7da8-4074-811a-860e76a6146a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.135883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c729ea5959e4581957ca5186718c804', 'request_id': 'req-cd994011-7da8-4074-811a-860e76a6146a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.179185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.179882 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.180163 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.180438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.181155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.183348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.201689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c729ea5959e4581957ca5186718c804', 'request_id': 'req-cd994011-7da8-4074-811a-860e76a6146a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.214433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] The token cache is being invalidate because project 7c729ea5959e4581957ca5186718c804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:01.214749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.215387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.216985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1146/2278] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:00 2022] DELETE /identity/v3/projects/7c729ea5959e4581957ca5186718c804 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:01.220918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.222225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:01.234371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd994011-7da8-4074-811a-860e76a6146a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.239289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.240215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.241241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.241241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] PATH_INFO: `/v3/users/74bb46a1f56c4026ab5f7702f8053f0b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.241941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74bb46a1f56c4026ab5f7702f8053f0b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.243887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.283015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1133/2279] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:46:01.298546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.298546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:01.299394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:01.356967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fb94f-25eb-4498-b490-417b3c647a3c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.362017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.362728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.363011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.363414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.364121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.366925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.463045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1134/2280] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:46:01.467564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.477786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a981ce-2a49-4620-a57d-2cac63717258 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.482405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.483188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.483474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.483755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.484481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.486472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.498668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1135/2281] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:01.501549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74bb46a1f56c4026ab5f7702f8053f0b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.504008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.516608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647fa0d-ca41-49eb-9727-0ac834311d8c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.519219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74bb46a1f56c4026ab5f7702f8053f0b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.522372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.522372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.522372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.522730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.523401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.525002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.535356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1136/2282] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:01.539839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.549323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a77830e-2e3f-40b3-8229-36bed48d842d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.555124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.561529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.561529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.561529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] PATH_INFO: `/v3/projects/a934b3068f7c4ecf8c91effe108dc198/users/c2ec4c69a4c845a5ba0f1f03c61a18f9/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.572289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a934b3068f7c4ecf8c91effe108dc198, user_id=c2ec4c69a4c845a5ba0f1f03c61a18f9, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.574978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.607941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74bb46a1f56c4026ab5f7702f8053f0b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.617444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74bb46a1f56c4026ab5f7702f8053f0b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.618202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-162d2434-cb18-486b-a302-f6373982bb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74bb46a1f56c4026ab5f7702f8053f0b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:01.624156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1147/2283] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:01 2022] DELETE /identity/v3/users/74bb46a1f56c4026ab5f7702f8053f0b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:01.682642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1137/2284] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:01 2022] PUT /identity/v3/projects/a934b3068f7c4ecf8c91effe108dc198/users/c2ec4c69a4c845a5ba0f1f03c61a18f9/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:01.693581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.694490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.694938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.725973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] MFA Rules not processed for user `c2ec4c69a4c845a5ba0f1f03c61a18f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:01.727297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:01.793148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1148/2285] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:46:01.798086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:01.811843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:01.871903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812661d5-286d-40db-be18-7e74e2eac6a4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:01.876791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:01.877460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:01.877745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:01.878021 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:01.878742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:01.880700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:01.981304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1138/2286] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:01.985714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.007230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e484e68-d685-47ee-bdbe-f8752b02bb5d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.075389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.076428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.076868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.077255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.077951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.079968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.171142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1149/2287] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:01 2022] POST /identity/v3/users => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:02.174493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.175050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.176260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.187879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bad48-9a86-4a78-9ff2-6e356f9cbb62 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.191388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3783097-3483-4f99-afc7-e3a55908cffe admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.197390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.198464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.198865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.199279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] PATH_INFO: `/v3/projects/50c14cf51638435a9759eee6fc673623` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.200509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c14cf51638435a9759eee6fc673623)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.202542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.205060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.205759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.206051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.206337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.207147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.209725 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.222035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1150/2288] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:02.226928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.239398 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee0772-aa3b-4e89-b690-8ecfacb82712 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.247952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.248915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.249203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.249481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] PATH_INFO: `/v3/projects/b4daacc7fda54649a97fd7e0e8574f70/users/e8fb616aea534b8988f0db4fe126102e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.266552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4daacc7fda54649a97fd7e0e8574f70, user_id=e8fb616aea534b8988f0db4fe126102e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.269232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.358277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1151/2289] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:02 2022] PUT /identity/v3/projects/b4daacc7fda54649a97fd7e0e8574f70/users/e8fb616aea534b8988f0db4fe126102e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:02.364020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.380960 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c14cf51638435a9759eee6fc673623', 'request_id': 'req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.443723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccd5a3-0419-4eba-a70d-38be8eace14f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.461470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c14cf51638435a9759eee6fc673623', 'request_id': 'req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.469359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c14cf51638435a9759eee6fc673623', 'request_id': 'req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.482627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] The token cache is being invalidate because project 50c14cf51638435a9759eee6fc673623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:02.482994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.483748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.485950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1139/2290] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:02 2022] DELETE /identity/v3/projects/50c14cf51638435a9759eee6fc673623 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:02.489629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.491844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.502557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.503271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.503554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.503827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.504550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.507350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.520002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1152/2291] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:02.527587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.529705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.545632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6aba66-a9cf-489f-9982-95bbb12db669 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.553157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.553157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.553157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.553157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] PATH_INFO: `/v3/projects/b4daacc7fda54649a97fd7e0e8574f70/users/e8fb616aea534b8988f0db4fe126102e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.555321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4daacc7fda54649a97fd7e0e8574f70, user_id=e8fb616aea534b8988f0db4fe126102e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.558214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8826a4-fe0e-4375-9858-4e3599b53b02 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.572377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1153/2292] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:02 2022] PUT /identity/v3/projects/b4daacc7fda54649a97fd7e0e8574f70/users/e8fb616aea534b8988f0db4fe126102e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:02.577638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f77f9-1f64-4aa9-b151-1baaaf1f2e01 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.585396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.585802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.586189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.620728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] MFA Rules not processed for user `e8fb616aea534b8988f0db4fe126102e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:02.622108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.655235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.656050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.656328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.656928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] PATH_INFO: `/v3/users/80655d49f93d4147b8b5006afb2a33c5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.659169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80655d49f93d4147b8b5006afb2a33c5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.661122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.706898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1154/2293] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:02.715411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:02.716818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.725430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:02.775870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e842a8-86fd-4cda-8bbc-e0d7880a1bf3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:02.779559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:02.780203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:02.780483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:02.780759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:02.782607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:02.839798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:02.841241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:02.855137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1155/2294] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:46:02.958765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80655d49f93d4147b8b5006afb2a33c5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:02.972855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80655d49f93d4147b8b5006afb2a33c5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:03.078666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80655d49f93d4147b8b5006afb2a33c5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:03.086266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80655d49f93d4147b8b5006afb2a33c5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:03.086956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-70397baa-6bf9-43da-aa4a-a1537a61cd42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80655d49f93d4147b8b5006afb2a33c5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:03.093346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1140/2295] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:02 2022] DELETE /identity/v3/users/80655d49f93d4147b8b5006afb2a33c5 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:03.551386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:03.606050 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:03.621569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d203d-9237-433a-bffc-1eb93275cc0d service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:03.737510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:03.739148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:03.739520 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:03.740657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] PATH_INFO: `/v3/projects/210150dcc63f40d6925991e2fdac9afb` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:03.743817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=210150dcc63f40d6925991e2fdac9afb)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:03.745879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:03.892777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210150dcc63f40d6925991e2fdac9afb', 'request_id': 'req-9e7b46df-3f24-4059-be49-2d7a36615f07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:03.900243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210150dcc63f40d6925991e2fdac9afb', 'request_id': 'req-9e7b46df-3f24-4059-be49-2d7a36615f07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:04.021682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210150dcc63f40d6925991e2fdac9afb', 'request_id': 'req-9e7b46df-3f24-4059-be49-2d7a36615f07'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:04.035393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] The token cache is being invalidate because project 210150dcc63f40d6925991e2fdac9afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:04.035989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:04.036669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9e7b46df-3f24-4059-be49-2d7a36615f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:04.038959 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1156/2296] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:03 2022] DELETE /identity/v3/projects/210150dcc63f40d6925991e2fdac9afb => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:04.044939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.045418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.045547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.064726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:04.065947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:04.130117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1141/2297] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:04.139063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.139501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.139870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.158422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:04.159659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:04.168273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1157/2298] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:04.173533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.182830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637df3f2-93f0-4524-883f-b810864f6bd3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.187521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.188170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.188479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.188769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.189482 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:04.189793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.191496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.203849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1142/2299] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:04.208190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.217465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abfe01-6c89-427d-be65-432fb9719391 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.222401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.223264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.223551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.223822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.225251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.227990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.334597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1158/2300] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:04 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:04.338660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.348665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8822-9e59-45f5-b252-3c66a95218c0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.351094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:04.406886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.407561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.407842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.408113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.408800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.410744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.529109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1143/2301] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:04 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:04.533297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.549020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefd6ce4-6961-48e0-aee5-a31b80d6a311 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.553979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.554745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.555007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.555287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.556037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.557678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.570762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1159/2302] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:04.574895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.586366 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b930a-8d88-47e2-8fef-cdbe64556296 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.592231 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.593124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.593501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.593925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.594826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.597116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c46a1e-075c-4bbc-b2e5-fce5764fa4e4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.613627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1144/2303] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:04.615098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.627477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12801090-bf02-4605-bb6c-c42253afea54 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.633856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.634894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.635180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.635466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] PATH_INFO: `/v3/projects/a413e0b4ab6c481da9f8bae2ae72249c/users/3c9c7010284e485eae20e901fec8dd16/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.653174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a413e0b4ab6c481da9f8bae2ae72249c, user_id=3c9c7010284e485eae20e901fec8dd16, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.655995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.733889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1160/2304] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:04 2022] PUT /identity/v3/projects/a413e0b4ab6c481da9f8bae2ae72249c/users/3c9c7010284e485eae20e901fec8dd16/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:04.746397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.746797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.747146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.779311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] MFA Rules not processed for user `3c9c7010284e485eae20e901fec8dd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:04.780588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:04.851859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1145/2305] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:04.864205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:04.866033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:04.941017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a602b-2703-45b8-9201-52303670a588 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:04.945812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:04.946253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:04.946470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:04.946783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:04.947062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:04.957097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:04.959494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:04.969254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1161/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:05.122249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:05.707437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:06.135930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:06.387837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:07.375906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:07.833421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:08.252628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:08.292938 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:08.447189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:08.501789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:08.724848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:16.603445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:16.604977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:16.618101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdaad7f-7525-48b3-8801-08ecd6550bad None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:16.622180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:16.622974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:16.630507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:16.630507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:16.630507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:16.658038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:16.692555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:16.697705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500a01a-33b0-4e9b-aae1-4cdf02735a07 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:16.704857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1146/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:18.945083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:20.298968 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:20.387270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f370f-9eee-40c7-b60a-da7b3da1171a service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:20.392009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:20.393000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:20.393370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:20.393708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] PATH_INFO: `/v3/users/3c9c7010284e485eae20e901fec8dd16` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:20.394998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9c7010284e485eae20e901fec8dd16)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:20.396991 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:20.666122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9c7010284e485eae20e901fec8dd16'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:20.678329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9c7010284e485eae20e901fec8dd16'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:20.852435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9c7010284e485eae20e901fec8dd16'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:20.860218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9c7010284e485eae20e901fec8dd16'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:20.860869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9c7010284e485eae20e901fec8dd16'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:20.866949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1162/2308] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:20 2022] DELETE /identity/v3/users/3c9c7010284e485eae20e901fec8dd16 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:20.871771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1505461-7360-4723-93b1-7cffe456f494 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:20.872259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1505461-7360-4723-93b1-7cffe456f494 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:20.872353 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1505461-7360-4723-93b1-7cffe456f494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:20.890244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a1505461-7360-4723-93b1-7cffe456f494 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:20.891488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a1505461-7360-4723-93b1-7cffe456f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:20.956634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1147/2309] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:20.966798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:20.968336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:21.027913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13f89a-1699-49c8-82ae-83abc6bd10e2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:21.032863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:21.033529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:21.033812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:21.034089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:21.044525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:21.045976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:21.055958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1163/2310] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:21.189955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1542-1136-4cc0-af6a-2c6009e0a1e9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:21.190252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1542-1136-4cc0-af6a-2c6009e0a1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:21.190566 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d1542-1136-4cc0-af6a-2c6009e0a1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:21.210613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f48d1542-1136-4cc0-af6a-2c6009e0a1e9 None None] MFA Rules not processed for user `7ec7931fdf144b3ba5f16970e0e2b00a`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:21.212496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f48d1542-1136-4cc0-af6a-2c6009e0a1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:21.269042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1148/2311] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:21.568035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:21.569669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:21.584106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e982d8-023d-4648-8441-b6cab82132d5 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:21.590800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:21.591737 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:21.594259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:21.594259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] PATH_INFO: `/v3/projects/a413e0b4ab6c481da9f8bae2ae72249c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:21.594259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] RBAC: Authorizing `identity:delete_project(project_id=a413e0b4ab6c481da9f8bae2ae72249c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:21.595575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:21.616854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:21.741687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a413e0b4ab6c481da9f8bae2ae72249c', 'request_id': 'req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:21.748064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a413e0b4ab6c481da9f8bae2ae72249c', 'request_id': 'req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:21.878664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a413e0b4ab6c481da9f8bae2ae72249c', 'request_id': 'req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:21.890507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] The token cache is being invalidate because project a413e0b4ab6c481da9f8bae2ae72249c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:21.890747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:21.891400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-93072f7e-9e57-48dd-b9e7-da3a2b57aeda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:21.893557 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1164/2312] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:21 2022] DELETE /identity/v3/projects/a413e0b4ab6c481da9f8bae2ae72249c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:21.899554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:21.900462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:21.900462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:21.919867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:21.921118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:21.988976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1149/2313] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:21.997744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:21.998114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:21.998457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.031772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:22.032984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:22.041786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1165/2314] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:22.045798 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.057212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d60a5-5ae6-4f88-8da2-7c15bb8bdecd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.061980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.062708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.063023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.063305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.064042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:22.064352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.065984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.074104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:22.079437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1150/2315] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:22.083912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.093844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0f144-e634-4a82-ab59-3afb8c40f0c0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.098592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.099236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.099513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.099785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.100462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.102576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.197461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1166/2316] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:22 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:22.201577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.212618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc8f0f-3ee6-42ab-aac1-defc51396e71 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.270621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.271305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.271587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.271864 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.272560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.275145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.379369 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1151/2317] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:22 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:22.386726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.397461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a45a7-c77b-420b-85ee-17d97a4d752f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.403579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.404403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.404779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.405151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.405893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.407613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.419945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1167/2318] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:22.424007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.434256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1888be-1ba8-4f33-8e68-1c1c947c5967 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.438922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.439573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.439852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.440126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.440868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.442553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c917157e-fe9c-4b1f-a7f7-5176edca697f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.455307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1152/2319] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:22.458171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.468647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb3e67-e0dd-4a2a-8dff-3b594cf85b61 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.473292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.474150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.474430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.474740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] PATH_INFO: `/v3/projects/bd87fd89fe8a4654be85fae12a22cfde/users/105a1ff916da4bb09aed5442bbaa35e1/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.490121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd87fd89fe8a4654be85fae12a22cfde, user_id=105a1ff916da4bb09aed5442bbaa35e1, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.492806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.583614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1168/2320] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:22 2022] PUT /identity/v3/projects/bd87fd89fe8a4654be85fae12a22cfde/users/105a1ff916da4bb09aed5442bbaa35e1/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:22.594166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.594453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.594754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.622600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] MFA Rules not processed for user `105a1ff916da4bb09aed5442bbaa35e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:22.623948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:22.692857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1153/2321] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:22.704958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:22.706585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:22.767174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7bdac-ad88-44f8-83fd-0be107a6f7c1 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:22.771003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:22.772603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:22.772603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:22.772603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:22.785416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:22.787117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7dcf18-6dd8-4627-87d1-b4ebad3bb3e8 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:22.799078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1169/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:24.116656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:24.118172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:24.188753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a34250-601d-4a83-8977-191f2114c7fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:24.193385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:24.194317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:24.194698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:24.194971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] PATH_INFO: `/v3/users/c2ec4c69a4c845a5ba0f1f03c61a18f9` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:24.196855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ec4c69a4c845a5ba0f1f03c61a18f9)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:24.199726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:24.205462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:24.396925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ec4c69a4c845a5ba0f1f03c61a18f9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:24.409903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ec4c69a4c845a5ba0f1f03c61a18f9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:24.496703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ec4c69a4c845a5ba0f1f03c61a18f9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:24.504629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ec4c69a4c845a5ba0f1f03c61a18f9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:24.505294 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ec4c69a4c845a5ba0f1f03c61a18f9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:24.512650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1154/2323] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:24 2022] DELETE /identity/v3/users/c2ec4c69a4c845a5ba0f1f03c61a18f9 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:24.517410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:24.517820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:24.518181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:24.537678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:24.539032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:24.583406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:24.602445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1170/2324] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:46:24.612614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:24.614104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:24.677719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b30190-17ee-43e0-942b-5dd07925061e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:24.682861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:24.683585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:24.683934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:24.684276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:24.693896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:24.695542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:24.707792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1155/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:46:25.248675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:25.252334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:25.265814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa227e-4b6b-43a7-9253-5cc819a9cdc4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:25.270682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:25.271546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:25.272354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:25.272634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] PATH_INFO: `/v3/projects/a934b3068f7c4ecf8c91effe108dc198` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:25.274261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a934b3068f7c4ecf8c91effe108dc198)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:25.276005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:25.391739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a934b3068f7c4ecf8c91effe108dc198', 'request_id': 'req-a80fb171-86fc-488e-b098-a34ebd5b3321'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.398389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a934b3068f7c4ecf8c91effe108dc198', 'request_id': 'req-a80fb171-86fc-488e-b098-a34ebd5b3321'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.479490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a934b3068f7c4ecf8c91effe108dc198', 'request_id': 'req-a80fb171-86fc-488e-b098-a34ebd5b3321'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.493224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] The token cache is being invalidate because project a934b3068f7c4ecf8c91effe108dc198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:25.493788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.498248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.501781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1171/2326] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:25 2022] DELETE /identity/v3/projects/a934b3068f7c4ecf8c91effe108dc198 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:25.505851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:25.516410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7c8-a36b-4405-b87d-5e8cb54d1225 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:25.577609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:25.578704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:25.579100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:25.579477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] PATH_INFO: `/v3/users/e8fb616aea534b8988f0db4fe126102e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:25.580995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fb616aea534b8988f0db4fe126102e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:25.583104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:25.838545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:25.852599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fb616aea534b8988f0db4fe126102e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.865219 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fb616aea534b8988f0db4fe126102e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.968791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fb616aea534b8988f0db4fe126102e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.976503 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fb616aea534b8988f0db4fe126102e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.977941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a5bf2849-1d1c-4c83-b4fa-3d18057d2e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fb616aea534b8988f0db4fe126102e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:25.984499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1156/2327] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:25 2022] DELETE /identity/v3/users/e8fb616aea534b8988f0db4fe126102e => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:26.517605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:26.519467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:26.537424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fb171-86fc-488e-b098-a34ebd5b3321 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:26.616146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:26.617075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:26.617380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:26.617654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] PATH_INFO: `/v3/projects/b4daacc7fda54649a97fd7e0e8574f70` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:26.619049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4daacc7fda54649a97fd7e0e8574f70)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:26.621077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:26.881874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4daacc7fda54649a97fd7e0e8574f70', 'request_id': 'req-e95b416f-6e26-4b84-bd06-0950bcf67272'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:26.896495 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4daacc7fda54649a97fd7e0e8574f70', 'request_id': 'req-e95b416f-6e26-4b84-bd06-0950bcf67272'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:26.970813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4daacc7fda54649a97fd7e0e8574f70', 'request_id': 'req-e95b416f-6e26-4b84-bd06-0950bcf67272'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:26.979428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:26.984435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] The token cache is being invalidate because project b4daacc7fda54649a97fd7e0e8574f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:26.984727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:26.985382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e95b416f-6e26-4b84-bd06-0950bcf67272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:26.987028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1172/2328] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:26 2022] DELETE /identity/v3/projects/b4daacc7fda54649a97fd7e0e8574f70 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:26.993023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:26.993477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:26.993663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.013086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:27.016377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:27.098840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1157/2329] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:27.109624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.110008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.110285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.131822 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:27.133377 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:27.143092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1173/2330] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:27.147735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.160972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6f7ca-7dab-4685-8fe1-afbbca6b2665 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.169318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.174067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.174751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.175330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.176101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:27.176415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.178208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.199967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1158/2331] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:27.204283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.217508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03f1a9-da97-4fe4-aa92-994a3d125856 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.224607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.225442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.225857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.227886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.227886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.229552 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.323334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1174/2332] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:27 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:27.324593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.337000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfc858-a163-4872-8f08-400e3f7ae738 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.457958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.458341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.458832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.459114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.459978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.463175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.631667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1159/2333] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:27 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:27.636885 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.647613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae72e9-b944-44b0-a310-57a059532b6e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.653111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.653782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.654068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.654349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.655232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.657127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.671902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1175/2334] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:27.677136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.688325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da235800-0705-4a2f-affa-c5825d4315f3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.693073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.693799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.694084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.694577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.696556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.698537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320f881-ff1c-4008-93d5-0b1e4bd7f178 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.717130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1160/2335] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:27.718571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:27.728941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8cd93-ce7b-4b41-a979-84dea7796f7f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:27.734305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:27.735501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.735876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.736576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] PATH_INFO: `/v3/projects/984d0e8eb2684f19846fc0625d89d082/users/e051f98d37924594b9db4a5c9c946a86/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.754002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984d0e8eb2684f19846fc0625d89d082, user_id=e051f98d37924594b9db4a5c9c946a86, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:27.757067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:27.857450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1176/2336] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:27 2022] PUT /identity/v3/projects/984d0e8eb2684f19846fc0625d89d082/users/e051f98d37924594b9db4a5c9c946a86/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:27.869603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:27.869889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:27.870220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:27.900417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:27.904207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] MFA Rules not processed for user `e051f98d37924594b9db4a5c9c946a86`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:27.905636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:27.984188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1161/2337] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:28.000075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:28.002506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:28.072305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434b643-3aba-40b8-b586-37e629d01d48 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:28.076129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:28.076784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:28.077072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:28.286370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:28.286370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:28.286370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a352f-2a59-403d-9e70-09e964edc392 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:28.286370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1177/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 14:46:28.287338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:28.361750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738932d6-945b-401f-8d1b-7dcf21b46797 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:28.366556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:28.368170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:28.368442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:28.368718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] PATH_INFO: `/v3/users/e051f98d37924594b9db4a5c9c946a86` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:28.370156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e051f98d37924594b9db4a5c9c946a86)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:28.372370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:28.739056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e051f98d37924594b9db4a5c9c946a86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:28.750880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e051f98d37924594b9db4a5c9c946a86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:28.995540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e051f98d37924594b9db4a5c9c946a86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:29.004626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e051f98d37924594b9db4a5c9c946a86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:29.005666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e051f98d37924594b9db4a5c9c946a86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:29.013575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1162/2339] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:28 2022] DELETE /identity/v3/users/e051f98d37924594b9db4a5c9c946a86 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:29.018010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809574-623c-4795-b004-ed41889312d6 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:29.018503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809574-623c-4795-b004-ed41889312d6 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:29.018652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f809574-623c-4795-b004-ed41889312d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:29.036982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8f809574-623c-4795-b004-ed41889312d6 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:29.038538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f809574-623c-4795-b004-ed41889312d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:29.102300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1178/2340] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:29.113226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:29.115251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:29.143861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:29.175359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef63041-3221-47f8-a732-89efc4bdb858 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:29.178900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:29.180082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:29.180356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:29.180626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:29.190414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:29.191906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d9c55f-2ad8-4adb-88ff-a809243b3008 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:29.203322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1163/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:29.687107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809574-623c-4795-b004-ed41889312d6 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:29.688510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f809574-623c-4795-b004-ed41889312d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:29.701289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f809574-623c-4795-b004-ed41889312d6 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:29.705955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:29.706849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:29.707134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:29.707613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] PATH_INFO: `/v3/projects/984d0e8eb2684f19846fc0625d89d082` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:29.712509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984d0e8eb2684f19846fc0625d89d082)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:29.712509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:29.913686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:29.928052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984d0e8eb2684f19846fc0625d89d082', 'request_id': 'req-f7db48ca-e168-4b69-8ef5-3898ceaa0590'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:29.935355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984d0e8eb2684f19846fc0625d89d082', 'request_id': 'req-f7db48ca-e168-4b69-8ef5-3898ceaa0590'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:30.044905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984d0e8eb2684f19846fc0625d89d082', 'request_id': 'req-f7db48ca-e168-4b69-8ef5-3898ceaa0590'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:30.055790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] The token cache is being invalidate because project 984d0e8eb2684f19846fc0625d89d082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:30.056076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:30.056699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f7db48ca-e168-4b69-8ef5-3898ceaa0590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:30.058244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1179/2342] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:29 2022] DELETE /identity/v3/projects/984d0e8eb2684f19846fc0625d89d082 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:30.063791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.064268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.064372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.082229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:30.083871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:30.148409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1164/2343] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:30.156703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.156988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.157267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.176388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:30.177634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:30.186010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1180/2344] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:46:30.190344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.201999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b87a55-b9ac-471e-b91a-9ec127cd9cde None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.207122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.207776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.208166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.208466 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.209186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:30.209489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.211202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.223703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1165/2345] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:30.228373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.238159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f8bdb-b13d-4e55-b033-642df8a84554 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.242935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.243936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.244106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.244389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.245082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.247003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.382056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1181/2346] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:46:30.386872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.397327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b179-4407-48fa-82e2-f1d2e35a9906 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.460113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.460797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.461084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.461359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.462647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.464626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.588916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1166/2347] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:30.592773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.603346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32052541-36a1-4cec-bbf4-28eed03f9fe5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.608380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.609063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.609348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.609625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.610359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.612052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.623548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1182/2348] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:30.627699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.640729 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e58d67-d3c5-4761-b9b6-4b60e5f24705 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.644906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.645577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.645860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.646170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.646991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.648628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d7f87b-8887-42e5-bec1-8a3073e41a24 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.659331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1167/2349] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:30.663376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:30.674097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb72cb-bb8e-47a9-b4aa-9093c135b1b9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:30.683063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:30.684150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.684432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.684731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] PATH_INFO: `/v3/projects/837e407b31f14e8f9f4faefb0535a171/users/5e3b5cd4fcbf4c66b9e3c0644101e1ad/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.700870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] RBAC: Authorizing `identity:create_grant(project_id=837e407b31f14e8f9f4faefb0535a171, user_id=5e3b5cd4fcbf4c66b9e3c0644101e1ad, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:30.703542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:30.851135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1183/2350] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:30 2022] PUT /identity/v3/projects/837e407b31f14e8f9f4faefb0535a171/users/5e3b5cd4fcbf4c66b9e3c0644101e1ad/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:30.862206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:30.862497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:30.862817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:30.892314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] MFA Rules not processed for user `5e3b5cd4fcbf4c66b9e3c0644101e1ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:30.893525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:30.913287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:30.969787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1168/2351] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:30.980579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.044553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbf9cf-1867-4b2b-a56d-830543e5971c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.049877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.050564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.050851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.051126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.051823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.053855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:31.218353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1184/2352] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:30 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:31.222490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.235837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111f63a-e530-4668-9e9a-779e3646087c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.296046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.296753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.297003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.297284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.298007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.300111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:31.466813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1169/2353] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:31 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:46:31.471152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.482452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69187b61-ff5a-403a-95b5-406ad1815fb2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.488117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.488842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.489124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.489396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.490122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.491992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:31.504831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1185/2354] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:31.509657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.520405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b56976-0adc-4146-a685-ff5aad6a339c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.525224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.525907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.526186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.526461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.527233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.528839 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24649fee-49e7-4a4c-a371-a1b65c32f072 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:31.540016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1170/2355] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:31.544273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.557546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838622a9-0235-4b41-92b6-45a4e55c9d68 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.559757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.560631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.560937 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.561213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] PATH_INFO: `/v3/projects/ff94fcb2a43549899e59b524ad6ae0b1/users/b79ecb9d998f44eb80b63d2d0d2f7558/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.577086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff94fcb2a43549899e59b524ad6ae0b1, user_id=b79ecb9d998f44eb80b63d2d0d2f7558, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.579794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:31.673107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1186/2356] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:31 2022] PUT /identity/v3/projects/ff94fcb2a43549899e59b524ad6ae0b1/users/b79ecb9d998f44eb80b63d2d0d2f7558/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:31.683802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.694813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.694813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.717873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] MFA Rules not processed for user `b79ecb9d998f44eb80b63d2d0d2f7558`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:31.720081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:31.793658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1171/2357] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:31.795618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:31.804742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:31.860592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a9941-027c-4dab-a452-ffe204ff5e7c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:31.865311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:31.865962 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:31.866279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:31.866570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:31.867262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:31.869305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.043460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1187/2358] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:31 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:32.047528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.059081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9a081-23e0-4bf4-91b5-eab79e0d2f23 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.088749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:32.127827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.130701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.130701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.130701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.130701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.131772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.262280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1172/2359] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:32 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:46:32.271642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.283232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0450d-e20d-4a1c-bf6a-7885544c7739 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.288125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.288831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.289115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.289392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.290141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.291805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.303479 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1188/2360] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:32.308325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.318262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fd7c7-8475-4325-94a2-2ea644103ff2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.323539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.324418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.324699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.324975 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] PATH_INFO: `/v3/projects/d50a6b65277643488155412f807a920b/users/1c2eae4fb79c4942a0a377c7fd4e7e4f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.342847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50a6b65277643488155412f807a920b, user_id=1c2eae4fb79c4942a0a377c7fd4e7e4f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.345428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.480239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1173/2361] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:32 2022] PUT /identity/v3/projects/d50a6b65277643488155412f807a920b/users/1c2eae4fb79c4942a0a377c7fd4e7e4f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:46:32.484519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.496339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d99c5-6129-42fe-9bac-87ea9e7ca493 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.562396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.563307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.563803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.564001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.564975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.566832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653ff9e-9da6-447d-8301-dbe60b0f0fb9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.578489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1189/2362] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:46:32.582906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.593855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47eb71-707d-4c1a-8b4a-abc31993a2e0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.603440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.603440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.603440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.603440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] PATH_INFO: `/v3/projects/d50a6b65277643488155412f807a920b/users/1c2eae4fb79c4942a0a377c7fd4e7e4f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.603832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50a6b65277643488155412f807a920b, user_id=1c2eae4fb79c4942a0a377c7fd4e7e4f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.604986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.615696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1174/2363] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:32 2022] PUT /identity/v3/projects/d50a6b65277643488155412f807a920b/users/1c2eae4fb79c4942a0a377c7fd4e7e4f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:32.627829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.628277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.628277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.659151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] MFA Rules not processed for user `1c2eae4fb79c4942a0a377c7fd4e7e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:32.660375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:32.733536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1190/2364] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:32.746326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:32.748148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:32.807504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef06a43-9554-4eac-a987-959ceb5cf4b6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:32.811248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:32.811943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:32.812211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:32.812524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:32.872932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:32.874409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:32.883354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1175/2365] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:36.164103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:36.165836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:36.173175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:36.186700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52366c75-fcc1-4493-9376-6e233a9f37e4 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:36.186700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:36.186700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:36.186700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:36.187087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:36.249147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:36.251121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:36.263457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1191/2366] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:42.879201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:42.891520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:42.900853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be521e3-275c-4eb5-8c22-34284a0438e4 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:42.904783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:42.911881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:42.911881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:42.911881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:42.917133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:42.918604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:42.929807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1176/2367] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:46.282911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=27 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:48.968595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:48.970020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:49.028117 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1d22b-247c-4629-86dd-f5238d6d3b27 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:49.031871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:49.032546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:49.032804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:49.033081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:49.036003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:49.102795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:49.104266 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628a986-eaaf-48f4-9279-ae938a3a26bd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:49.114342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1192/2368] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:51.082027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:51.152359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531db35-ef36-4208-85e7-6965a56a4678 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:51.156802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:51.157724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:51.158046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:51.158313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] PATH_INFO: `/v3/users/5e3b5cd4fcbf4c66b9e3c0644101e1ad` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:51.160335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3b5cd4fcbf4c66b9e3c0644101e1ad)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:51.162265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:51.464035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3b5cd4fcbf4c66b9e3c0644101e1ad'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:51.478540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3b5cd4fcbf4c66b9e3c0644101e1ad'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:51.625170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3b5cd4fcbf4c66b9e3c0644101e1ad'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:51.634025 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3b5cd4fcbf4c66b9e3c0644101e1ad'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:51.634757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3b5cd4fcbf4c66b9e3c0644101e1ad'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:51.641218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1177/2369] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:51 2022] DELETE /identity/v3/users/5e3b5cd4fcbf4c66b9e3c0644101e1ad => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:51.647692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:51.648240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:51.648460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:51.671535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:51.673105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:51.761738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1193/2370] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:51.771730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:51.773239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:51.847118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fec340-c553-4725-a553-ff2cccfe8e7f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:51.850968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:51.851754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:51.852141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:51.852527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:51.863599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:51.865285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:51.876490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1178/2371] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:46:52.442930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:52.444372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:52.460180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01ada-c441-4fb8-8f43-b654732dc2fa None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:52.465086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:52.466115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:52.466401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:52.466723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] PATH_INFO: `/v3/projects/837e407b31f14e8f9f4faefb0535a171` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:52.467911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=837e407b31f14e8f9f4faefb0535a171)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:52.470011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:52.597582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:52.600570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '837e407b31f14e8f9f4faefb0535a171', 'request_id': 'req-0a467018-3fd3-4365-8255-659eec8fb131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:52.607595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '837e407b31f14e8f9f4faefb0535a171', 'request_id': 'req-0a467018-3fd3-4365-8255-659eec8fb131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:52.700509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '837e407b31f14e8f9f4faefb0535a171', 'request_id': 'req-0a467018-3fd3-4365-8255-659eec8fb131'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:52.712440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] The token cache is being invalidate because project 837e407b31f14e8f9f4faefb0535a171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:52.712732 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:52.713403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:52.715040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1194/2372] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:52 2022] DELETE /identity/v3/projects/837e407b31f14e8f9f4faefb0535a171 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:52.719006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:52.734156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd42bc5-572a-46d0-9d96-9b95593b6d8d service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:52.739819 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:52.792877 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:52.793777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:52.794055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:52.794328 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] PATH_INFO: `/v3/users/b79ecb9d998f44eb80b63d2d0d2f7558` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:52.795676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b79ecb9d998f44eb80b63d2d0d2f7558)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:52.797663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:53.040282 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b79ecb9d998f44eb80b63d2d0d2f7558'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.053051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b79ecb9d998f44eb80b63d2d0d2f7558'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.239885 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b79ecb9d998f44eb80b63d2d0d2f7558'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.247313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b79ecb9d998f44eb80b63d2d0d2f7558'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.247988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b79ecb9d998f44eb80b63d2d0d2f7558'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.255176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1179/2373] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:52 2022] DELETE /identity/v3/users/b79ecb9d998f44eb80b63d2d0d2f7558 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:53.667912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:53.669411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:53.683221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a467018-3fd3-4365-8255-659eec8fb131 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:53.750062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:53.751058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:53.751356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:53.751652 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] PATH_INFO: `/v3/projects/ff94fcb2a43549899e59b524ad6ae0b1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:53.753355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff94fcb2a43549899e59b524ad6ae0b1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:53.755376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:53.855961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff94fcb2a43549899e59b524ad6ae0b1', 'request_id': 'req-76928ee4-e422-48e5-a79b-35028ba66b8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.862655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff94fcb2a43549899e59b524ad6ae0b1', 'request_id': 'req-76928ee4-e422-48e5-a79b-35028ba66b8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.941829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff94fcb2a43549899e59b524ad6ae0b1', 'request_id': 'req-76928ee4-e422-48e5-a79b-35028ba66b8a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.953888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] The token cache is being invalidate because project ff94fcb2a43549899e59b524ad6ae0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:53.954170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.954844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:53.956506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1195/2374] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:53 2022] DELETE /identity/v3/projects/ff94fcb2a43549899e59b524ad6ae0b1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:53.960825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:53.962268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:53.975127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26939c89-e8cb-4ff1-a6f9-ef27a452b520 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:54.036352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:54.037228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:54.037505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:54.037778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] PATH_INFO: `/v3/users/1c2eae4fb79c4942a0a377c7fd4e7e4f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:54.039012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2eae4fb79c4942a0a377c7fd4e7e4f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:54.040972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:54.221805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:54.235952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2eae4fb79c4942a0a377c7fd4e7e4f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:54.249831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2eae4fb79c4942a0a377c7fd4e7e4f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:54.318314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2eae4fb79c4942a0a377c7fd4e7e4f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:54.325818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2eae4fb79c4942a0a377c7fd4e7e4f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:54.326475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6aac69a5-60a5-4137-955e-59129ba4493f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2eae4fb79c4942a0a377c7fd4e7e4f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:54.333116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1180/2375] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:46:53 2022] DELETE /identity/v3/users/1c2eae4fb79c4942a0a377c7fd4e7e4f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:54.768395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:54.770841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:54.784461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76928ee4-e422-48e5-a79b-35028ba66b8a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:54.846528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:54.847998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:54.848286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:54.848564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] PATH_INFO: `/v3/projects/d50a6b65277643488155412f807a920b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:54.849983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50a6b65277643488155412f807a920b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:54.852200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:54.860252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:55.026468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50a6b65277643488155412f807a920b', 'request_id': 'req-45bf3def-8dbb-4cf1-b362-936b13e4fc15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:55.032619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50a6b65277643488155412f807a920b', 'request_id': 'req-45bf3def-8dbb-4cf1-b362-936b13e4fc15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:55.104132 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50a6b65277643488155412f807a920b', 'request_id': 'req-45bf3def-8dbb-4cf1-b362-936b13e4fc15'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:55.117763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] The token cache is being invalidate because project d50a6b65277643488155412f807a920b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:46:55.118057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:55.118766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-45bf3def-8dbb-4cf1-b362-936b13e4fc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:46:55.120346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1196/2376] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:46:54 2022] DELETE /identity/v3/projects/d50a6b65277643488155412f807a920b => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:55.125401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.125869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.125997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.144501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:55.145684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:55.205119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1181/2377] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:55.213452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.213738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.214011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.233638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:55.235020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:55.243703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1197/2378] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:55.248611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.259625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9f28a-8f08-4fbc-aa48-56b6a1da86dc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.264406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.265075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.265356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.265632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.266389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:46:55.266727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.268360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.280609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1182/2379] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:46:55.285715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.296222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f6307-9b84-4dc3-a2a6-938c4e4c6d6d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.302246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.302935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.303215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.303487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.304188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.306093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.393420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1198/2380] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:46:55 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:55.398020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.408590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f03e1-7cd1-44f3-9a4d-b745f874b515 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.474538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.475234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.475540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.475821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.476506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.478432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.592723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1183/2381] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:46:55 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:46:55.596749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.608141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16115d9d-ed5f-4466-b5f8-f705ba538bf3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.613201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.613854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.614131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.614430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.615178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.616761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.627680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1199/2382] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:55.633458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.644517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bdc80-a1c4-4cb3-8853-99d4be6fc050 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.651063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.651726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.652006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.652278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.653091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.654806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147184e-b059-4b42-965e-0d3734abbf8e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.666375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1184/2383] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:46:55.670468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.681380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099e4e2-1f36-4f33-9d30-c88f4c9e6c64 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:55.687086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:55.688379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.688662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.688939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] PATH_INFO: `/v3/projects/13ef5e3ad7f94b8f87a7172c8d8792da/users/bd26991481f141d8b95c9981d93d0c14/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.706775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13ef5e3ad7f94b8f87a7172c8d8792da, user_id=bd26991481f141d8b95c9981d93d0c14, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:55.709452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:55.825115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1200/2384] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:46:55 2022] PUT /identity/v3/projects/13ef5e3ad7f94b8f87a7172c8d8792da/users/bd26991481f141d8b95c9981d93d0c14/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:46:55.835444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:55.835738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:55.836022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:55.861659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:46:55.865660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] MFA Rules not processed for user `bd26991481f141d8b95c9981d93d0c14`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:46:55.866941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:55.934972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1185/2385] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:46:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:46:55.946609 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:46:55.948344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:46:56.007752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3121-5d4e-438f-857b-9d6c5e8fab74 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:46:56.011575 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:46:56.012224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:46:56.012502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:46:56.012781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:46:56.021913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:46:56.023379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:46:56.032167 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1201/2386] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:46:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:47:06.028048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:06.151232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=26 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:22.352092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:22.353639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:22.448281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726652f-347c-4eca-ad90-20a4065eea6a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:22.453286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:22.453934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:22.454300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:22.454695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:22.456481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:22.467779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:22.469367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d129b9a8-2e89-4590-962f-76d556ab80d0 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:22.479198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1186/2387] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:47:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:47:32.475039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=41 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:45.855217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:45.923321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d74c5-8217-4a5b-88a0-45fef7d95ebf service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:45.924152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:45.925031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:45.925310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:45.925584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] PATH_INFO: `/v3/users/bd26991481f141d8b95c9981d93d0c14` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:45.926893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd26991481f141d8b95c9981d93d0c14)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:45.928866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:46.140492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd26991481f141d8b95c9981d93d0c14'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:46.153541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd26991481f141d8b95c9981d93d0c14'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:46.228440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd26991481f141d8b95c9981d93d0c14'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:46.236531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd26991481f141d8b95c9981d93d0c14'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:46.237273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd26991481f141d8b95c9981d93d0c14'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:46.245608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1202/2388] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:47:45 2022] DELETE /identity/v3/users/bd26991481f141d8b95c9981d93d0c14 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:47:46.250239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:46.250771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:46.250889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:46.270319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:46.271979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:46.332527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1187/2389] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:46.342750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:46.344187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:46.407536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b4971-8382-4519-9527-83a5e328188d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:46.411430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:46.412082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:46.415080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:46.415080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:46.422009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:46.423485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46619a-36b5-4675-9603-2fd1551c4038 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:46.433853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1203/2390] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:47:46.892261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:46.893752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:46.906336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4e5eb-fb2f-43a1-8a0e-4709ac8eecc8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:46.918451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:46.919643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:46.919981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:46.920279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] PATH_INFO: `/v3/projects/13ef5e3ad7f94b8f87a7172c8d8792da` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:46.921517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13ef5e3ad7f94b8f87a7172c8d8792da)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:46.923829 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:47.108061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13ef5e3ad7f94b8f87a7172c8d8792da', 'request_id': 'req-0e9d8334-c15b-43a5-ac77-2208534c707a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:47.176223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13ef5e3ad7f94b8f87a7172c8d8792da', 'request_id': 'req-0e9d8334-c15b-43a5-ac77-2208534c707a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:47.183399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13ef5e3ad7f94b8f87a7172c8d8792da', 'request_id': 'req-0e9d8334-c15b-43a5-ac77-2208534c707a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:47.194929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] The token cache is being invalidate because project 13ef5e3ad7f94b8f87a7172c8d8792da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:47:47.195218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:47.195981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:47.197572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1188/2391] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:47:46 2022] DELETE /identity/v3/projects/13ef5e3ad7f94b8f87a7172c8d8792da => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 14:47:48.428551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2c333-3be2-491c-ac8b-d1598b2a1067 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:48.429005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2c333-3be2-491c-ac8b-d1598b2a1067 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:48.429122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2c333-3be2-491c-ac8b-d1598b2a1067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:48.447636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-32d2c333-3be2-491c-ac8b-d1598b2a1067 None None] MFA Rules not processed for user `ea0f85ae8bd942338bfcda177b6864a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:48.448788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-32d2c333-3be2-491c-ac8b-d1598b2a1067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:48.469254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:48.508599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1204/2392] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:55.103039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:55.105005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:55.174715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d8334-c15b-43a5-ac77-2208534c707a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:55.184643 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:55.185740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:55.186119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:55.186533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] PATH_INFO: `/v3/users/105a1ff916da4bb09aed5442bbaa35e1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:55.187872 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105a1ff916da4bb09aed5442bbaa35e1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:55.190028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:55.268177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:55.344736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105a1ff916da4bb09aed5442bbaa35e1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:55.371141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105a1ff916da4bb09aed5442bbaa35e1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:55.445100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105a1ff916da4bb09aed5442bbaa35e1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:55.452593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105a1ff916da4bb09aed5442bbaa35e1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:55.453242 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105a1ff916da4bb09aed5442bbaa35e1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:55.459447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1189/2393] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:47:55 2022] DELETE /identity/v3/users/105a1ff916da4bb09aed5442bbaa35e1 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:47:55.464078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:55.464562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:55.464658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:55.483219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:55.484431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:55.544825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1205/2394] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:55.555075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:55.556525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:55.614281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12727803-7a91-4dba-8c6f-daf5893205d6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:55.618639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:55.619283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:55.619606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:55.619879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:55.628898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:55.630472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719ba4-9ade-431f-9433-32aed51d0636 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:55.639870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1190/2395] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:47:56.080063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.081529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:56.096380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1adc8c-4628-4c59-980c-ba86f2c33dd5 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.102734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.103604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.104014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.104302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] PATH_INFO: `/v3/projects/bd87fd89fe8a4654be85fae12a22cfde` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.105500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd87fd89fe8a4654be85fae12a22cfde)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.116662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.280607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd87fd89fe8a4654be85fae12a22cfde', 'request_id': 'req-cc0983df-d69e-448a-9373-1376b27b5968'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:56.288437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd87fd89fe8a4654be85fae12a22cfde', 'request_id': 'req-cc0983df-d69e-448a-9373-1376b27b5968'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:56.409957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd87fd89fe8a4654be85fae12a22cfde', 'request_id': 'req-cc0983df-d69e-448a-9373-1376b27b5968'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:56.422114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] The token cache is being invalidate because project bd87fd89fe8a4654be85fae12a22cfde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:47:56.422506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:56.423313 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-cc0983df-d69e-448a-9373-1376b27b5968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:47:56.425129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1206/2396] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:47:56 2022] DELETE /identity/v3/projects/bd87fd89fe8a4654be85fae12a22cfde => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:47:56.435945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.436788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.437087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.455391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:56.456572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:56.514876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1191/2397] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:56.523124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.523581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.523696 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.541525 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:56.542748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:56.550486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1207/2398] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:56.554734 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.564430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb777ae-56fb-4e8d-bd6f-034a3ced4419 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.569615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.570263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.570573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.570853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.571572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:47:56.571879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.573555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.585896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1192/2399] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:47:56.590028 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.599445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccad62-840b-4212-9ad3-08c1fd6d939e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.604429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.605079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.605364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.605648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.606341 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.608287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.719421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1208/2400] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:47:56 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:47:56.723454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.733893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331ddab-92ba-4b8a-8d53-663410cd3fe3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.765634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:56.794097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.794801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.795078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.795348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.796047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.798001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.885111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1193/2401] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:47:56 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:47:56.889074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.899266 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d616d-32cb-4ba9-b0f4-bfc502ae7813 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.903923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.904581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.904863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.905139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.905866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.907507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.918399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1209/2402] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 14:47:56.922365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.933090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe3cce-9ab3-4be2-bb02-fe40be5ade17 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.937752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.938402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.938715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.938994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.939718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.941364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aabae2-ae22-48ce-8b59-70800f14768b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:56.952092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1194/2403] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:47:56.955997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:56.965762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aeadbc-6a3a-4777-96cd-0563229120ae admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:56.970335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:56.971231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:56.971508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:56.971775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] PATH_INFO: `/v3/projects/3b4280ef123a47419104a9e4c15aae30/users/e3c22425bb9845ada20e1a859142a32d/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:56.980672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:47:56.987423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4280ef123a47419104a9e4c15aae30, user_id=e3c22425bb9845ada20e1a859142a32d, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:56.990806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:57.061494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1210/2404] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:47:56 2022] PUT /identity/v3/projects/3b4280ef123a47419104a9e4c15aae30/users/e3c22425bb9845ada20e1a859142a32d/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:47:57.074081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba15f-1f18-477d-b3c2-91a42dc98550 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:57.074499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba15f-1f18-477d-b3c2-91a42dc98550 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:57.074812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ba15f-1f18-477d-b3c2-91a42dc98550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:57.110589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-448ba15f-1f18-477d-b3c2-91a42dc98550 None None] MFA Rules not processed for user `e3c22425bb9845ada20e1a859142a32d`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:47:57.111843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-448ba15f-1f18-477d-b3c2-91a42dc98550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:57.179361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1195/2405] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:47:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:47:57.190206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:47:57.191927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:47:57.251596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb19ae-2694-4fe3-b4c4-00d3c76549f8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:47:57.255276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:47:57.255916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:47:57.256195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:47:57.256468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:47:57.264742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:47:57.266163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:47:57.274884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1211/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:47:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:48:00.016453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfa60-9763-488e-b8a4-c4a295cefcdb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:00.016978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfa60-9763-488e-b8a4-c4a295cefcdb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:00.017731 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfa60-9763-488e-b8a4-c4a295cefcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:00.036097 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-aebcfa60-9763-488e-b8a4-c4a295cefcdb None None] MFA Rules not processed for user `e3c22425bb9845ada20e1a859142a32d`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:00.037150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aebcfa60-9763-488e-b8a4-c4a295cefcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:00.044174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1196/2407] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:48:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:07.270953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=31,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:10.030444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:18.132709 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:18.134124 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:18.190105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260ff3-1ef2-4234-a182-bffe208db900 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:18.194227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:18.194887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:18.195171 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:18.195448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:18.197213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:18.266671 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:18.268110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:18.280589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1212/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:48:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:48:19.328485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5961-728d-4d0c-ad1f-2ac0e5500596 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:19.329051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5961-728d-4d0c-ad1f-2ac0e5500596 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:19.329283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5961-728d-4d0c-ad1f-2ac0e5500596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:19.352895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-126b5961-728d-4d0c-ad1f-2ac0e5500596 None None] MFA Rules not processed for user `218c2788d21f4912822841185afbfaf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:19.358506 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-126b5961-728d-4d0c-ad1f-2ac0e5500596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:19.419474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1197/2409] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:48:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:19.453898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:19.455311 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:19.534931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52a49a-fcdd-4c09-aacf-3894db8b22ff service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:19.539323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:19.540113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:19.540395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:19.540667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:19.542501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:19.557942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:19.559542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:19.570899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1213/2410] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:48:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:48:29.411874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=25 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:29.565883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:29.614834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4137-59ec-4894-b706-eeac19ca39d4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:29.615126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4137-59ec-4894-b706-eeac19ca39d4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:29.615410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4137-59ec-4894-b706-eeac19ca39d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:29.677015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a9de4137-59ec-4894-b706-eeac19ca39d4 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:29.678243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a9de4137-59ec-4894-b706-eeac19ca39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:29.748688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1198/2411] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:48:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:29.793487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:29.795133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:29.888289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c01141-9109-4940-a27f-f7b9d480a6ba service glance] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:29.893384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:29.894129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:29.894472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:29.894779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:29.904137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:29.905790 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:29.914447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1214/2412] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:48:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:48:30.765874 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348c318-effd-41d5-818f-87e9c1fd42da None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:30.766299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348c318-effd-41d5-818f-87e9c1fd42da None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:30.766409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348c318-effd-41d5-818f-87e9c1fd42da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:30.787069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5348c318-effd-41d5-818f-87e9c1fd42da None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:30.788446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5348c318-effd-41d5-818f-87e9c1fd42da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:30.795955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1199/2413] 10.222.0.23 () {58 vars in 967 bytes} [Fri Jun 24 14:48:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:30.809335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:30.819986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c226ca-c61a-4bb9-81c5-b7ce03e0fdfa service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:30.825777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:30.826421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:30.826736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:30.827020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:30.835267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:30.836932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:30.845496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1215/2414] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:48:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:48:31.520350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626a9c-cd0a-4d67-8fe2-361592fb6247 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:31.520635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626a9c-cd0a-4d67-8fe2-361592fb6247 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:31.520911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626a9c-cd0a-4d67-8fe2-361592fb6247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:31.538284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c3626a9c-cd0a-4d67-8fe2-361592fb6247 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:31.539565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c3626a9c-cd0a-4d67-8fe2-361592fb6247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:31.546740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1200/2415] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 14:48:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:31.558884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:31.569254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff70d7-ef76-4062-aa32-7411042ade09 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:31.575030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:31.575660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:31.575944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:31.576218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:31.586040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:31.622077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:31.622077 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1216/2416] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:48:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 14:48:32.263378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:32.263863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:32.263976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:32.281648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:32.282860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:32.289840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1201/2417] 10.222.0.23 () {58 vars in 967 bytes} [Fri Jun 24 14:48:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:32.302225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:32.312707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba205147-e081-49f2-9557-45bf1169f0b3 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:32.317710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:32.318286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:32.318667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:32.318948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:32.327048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:32.328663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d470f-83b0-43b6-b9e1-0c54a5f5e370 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:32.336889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1217/2418] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 14:48:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 14:48:35.004812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:35.070662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d365c4-e99e-4568-a9ee-d477f6f1c2fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:35.076435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:35.077333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:35.077784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:35.078065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] PATH_INFO: `/v3/users/e3c22425bb9845ada20e1a859142a32d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:35.079285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c22425bb9845ada20e1a859142a32d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:35.081343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:35.287585 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:35.289151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c22425bb9845ada20e1a859142a32d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.301785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c22425bb9845ada20e1a859142a32d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.389707 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c22425bb9845ada20e1a859142a32d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.397768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c22425bb9845ada20e1a859142a32d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.398541 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c22425bb9845ada20e1a859142a32d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.408071 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1202/2419] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:48:35 2022] DELETE /identity/v3/users/e3c22425bb9845ada20e1a859142a32d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:48:35.410773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:35.410773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:35.410773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:35.429804 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:48:35.431066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:35.496369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1218/2420] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:48:35.508036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:35.509455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:35.575310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ea6a2-2cfb-46b1-a666-445afb7e64a0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:35.575310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:35.575754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:35.575754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:35.575932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:35.585350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:35.586964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:35.598117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1203/2421] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:48:35.836918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:48:35.838356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:48:35.851160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec51ed0-53e6-4a98-b632-663bd8c87d56 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:48:35.857256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:48:35.858125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:48:35.858411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:48:35.858719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] PATH_INFO: `/v3/projects/3b4280ef123a47419104a9e4c15aae30` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:48:35.859927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4280ef123a47419104a9e4c15aae30)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:48:35.861939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:48:35.928101 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:35.943812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4280ef123a47419104a9e4c15aae30', 'request_id': 'req-38b26151-aed1-4605-99f0-32700905be67'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:35.950074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4280ef123a47419104a9e4c15aae30', 'request_id': 'req-38b26151-aed1-4605-99f0-32700905be67'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:36.006651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4280ef123a47419104a9e4c15aae30', 'request_id': 'req-38b26151-aed1-4605-99f0-32700905be67'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:36.017497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] The token cache is being invalidate because project 3b4280ef123a47419104a9e4c15aae30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:48:36.018066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:36.019226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:48:36.020783 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1219/2422] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:48:35 2022] DELETE /identity/v3/projects/3b4280ef123a47419104a9e4c15aae30 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:48:45.593646 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:48:46.019350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:49:13.578831 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:49:13.580468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:13.656472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ed738-6976-4552-9b73-e633d88c634b service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:49:13.719129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:49:13.719782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:13.720063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:13.720377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:13.722382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:13.734752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:49:13.736347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:49:13.747432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1204/2423] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:49:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:49:22.516222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:49:22.517685 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:22.533635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b26151-aed1-4605-99f0-32700905be67 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:49:22.539243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:49:22.540409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:22.540681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:22.540953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:22.542707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:22.606907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:49:22.608592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:49:22.617901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1220/2424] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:49:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:49:23.743763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=62 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:49:25.272129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:49:25.273649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:25.330458 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1df5a3-7131-4592-941a-5d10ac5b7653 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:49:25.334249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:49:25.334928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:25.335232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:25.335507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:25.337444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:25.347487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:49:25.348915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:49:25.361522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1205/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:49:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:49:32.613540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:49:35.355150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=26 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:49:48.559566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:49:48.561203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:48.627295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab4daa-fb51-4cbb-bee2-ae034d178837 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:49:48.631828 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:49:48.632695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:48.632975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:48.633247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] PATH_INFO: `/v3/users/218c2788d21f4912822841185afbfaf0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:48.634425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=218c2788d21f4912822841185afbfaf0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:49:48.636476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:49:48.854372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218c2788d21f4912822841185afbfaf0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:48.866022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218c2788d21f4912822841185afbfaf0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:48.939480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218c2788d21f4912822841185afbfaf0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:48.947098 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218c2788d21f4912822841185afbfaf0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:48.947753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1f3e975-ff97-495a-bcb0-6926558739fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218c2788d21f4912822841185afbfaf0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:48.954788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1221/2426] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:49:48 2022] DELETE /identity/v3/users/218c2788d21f4912822841185afbfaf0 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:49:49.204515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:49:49.216595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa2d5b-e481-4748-a787-1f19b8dcd696 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:49:49.274376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:49:49.275256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:49.275537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:49.275807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] PATH_INFO: `/v3/projects/5bd07538140e49dc80b0d61a1a404754` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:49.277149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd07538140e49dc80b0d61a1a404754)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:49:49.279096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:49:49.381250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd07538140e49dc80b0d61a1a404754', 'request_id': 'req-b620c3d9-2c08-4e9b-823f-bcec176340cd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:49.453903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd07538140e49dc80b0d61a1a404754', 'request_id': 'req-b620c3d9-2c08-4e9b-823f-bcec176340cd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:49.460855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd07538140e49dc80b0d61a1a404754', 'request_id': 'req-b620c3d9-2c08-4e9b-823f-bcec176340cd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:49.472006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] The token cache is being invalidate because project 5bd07538140e49dc80b0d61a1a404754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:49:49.472298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:49.472951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:49:49.474823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1206/2427] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:49:49 2022] DELETE /identity/v3/projects/5bd07538140e49dc80b0d61a1a404754 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:49:57.640437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb56d2-6531-4c0f-836b-5a0c6d76efdb None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:49:57.640955 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb56d2-6531-4c0f-836b-5a0c6d76efdb None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:49:57.641053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb56d2-6531-4c0f-836b-5a0c6d76efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:49:57.659589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d7cb56d2-6531-4c0f-836b-5a0c6d76efdb None None] MFA Rules not processed for user `174b3629771b4044bb7b1cee523225d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:49:57.660788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d7cb56d2-6531-4c0f-836b-5a0c6d76efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:49:57.716201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1222/2428] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:49:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:49:59.468673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:06.197912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:06.199287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:06.272118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620c3d9-2c08-4e9b-823f-bcec176340cd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:06.277591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:06.278232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:06.278509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:06.278845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:06.280762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:06.337657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:06.339292 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:06.348610 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1207/2429] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:07.709127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:16.344431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=56 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:26.000814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:26.001303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:26.001408 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:26.018618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] MFA Rules not processed for user `7ec7931fdf144b3ba5f16970e0e2b00a`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:50:26.019827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:26.073793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1223/2430] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:50:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:50:26.382075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:26.383554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:26.439335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86a6ec-229e-42ee-b334-c397bee3d6d2 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:26.443035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:26.443676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:26.443960 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:26.444233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:26.446023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:26.455315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:26.456759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:26.464618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1208/2431] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:27.124446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:27.125872 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:27.198622 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446c416-5216-43a2-a10c-52a1e3cf2065 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:27.204309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:27.204982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:27.205264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:27.205540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:27.207364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:27.219029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:27.220604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:27.230534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1224/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:31.370276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:31.374236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:31.396677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579dd0b-9fa3-48fe-aa3d-a34ac8021ee2 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:31.407724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:31.407724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:31.407724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:31.407724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:31.408180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:31.422165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:31.424003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:31.437528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1209/2433] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:37.226152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=57 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:39.362033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:39.374129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1027f1-8f35-4f01-9a53-ec6e1eb8aa75 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:39.379713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:39.380380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:39.380659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:39.380930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:39.382705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:39.439642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:39.441214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:39.450274 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1225/2434] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:41.431512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:44.357222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:44.372928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c95ca-cea6-452f-91ca-eadc3223fdc0 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:44.379914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:44.380635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:44.380879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:44.381156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:44.383699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:44.397085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:44.398777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55667a-7f7a-489a-a3b6-4903d9917207 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:44.411486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1210/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:45.621029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:50:45.633866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a95f-8522-4fb5-9c53-8c8efbb7463c service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:50:45.639535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:50:45.640175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:45.640457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:45.640868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:45.642712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:45.653719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:50:45.655369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:50:45.664899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1226/2436] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:50:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:50:54.406474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:55.660499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=34 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:50:57.451809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e36b5-72c6-4d1d-adfa-eba8994019eb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:50:57.452286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e36b5-72c6-4d1d-adfa-eba8994019eb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:50:57.452381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8e36b5-72c6-4d1d-adfa-eba8994019eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:50:57.473303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2e8e36b5-72c6-4d1d-adfa-eba8994019eb None None] MFA Rules not processed for user `ea0f85ae8bd942338bfcda177b6864a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:50:57.474553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2e8e36b5-72c6-4d1d-adfa-eba8994019eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:50:57.481551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1211/2437] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:50:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:51:07.466895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:51:41.613996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:41.615449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:41.627286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8312264-d428-45b6-8768-ed6565351941 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:41.632649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:41.633302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:41.633585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:41.633859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:41.635710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:41.702465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:41.703946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131c3ca-ea88-4b9b-b135-0d5624eb0a74 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:41.715190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1227/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:51:45.449122 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63850-5952-4c66-955c-388e5502a840 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:45.449976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63850-5952-4c66-955c-388e5502a840 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:45.450287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63850-5952-4c66-955c-388e5502a840 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:45.452171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1212/2439] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 14:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 14:51:45.462994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:45.463534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:45.463534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:45.503711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:45.504992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:45.514284 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1228/2440] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 14:51:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:51:45.525952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63850-5952-4c66-955c-388e5502a840 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:45.540188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63850-5952-4c66-955c-388e5502a840 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:45.545399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:45.546151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:45.546508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:45.546886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:45.556213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:45.557945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:45.568562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1213/2441] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:51:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:51:45.605466 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:45.606951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:45.621523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d14d25-9d51-4a12-9c01-d3f89c95f856 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:45.626281 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:45.627176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:45.627456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:45.627731 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] PATH_INFO: `/v3/users/174b3629771b4044bb7b1cee523225d1` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:45.629514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=174b3629771b4044bb7b1cee523225d1)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:45.631639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:45.839445 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174b3629771b4044bb7b1cee523225d1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:45.852111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174b3629771b4044bb7b1cee523225d1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:45.940583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174b3629771b4044bb7b1cee523225d1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:45.948999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174b3629771b4044bb7b1cee523225d1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:45.949711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-781c8e4a-bbd9-499f-95dd-9ae16bf7b7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174b3629771b4044bb7b1cee523225d1'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:45.955800 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1229/2442] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 14:51:45 2022] DELETE /identity/v3/users/174b3629771b4044bb7b1cee523225d1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:51:46.211444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:46.224588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188891-b111-4e6d-9d42-7d375d9d8769 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:46.284207 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:46.285673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:46.285958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:46.286257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] PATH_INFO: `/v3/projects/0abdd49efea54add810ad0668df4a899` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:46.287493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0abdd49efea54add810ad0668df4a899)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:46.289427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:46.427786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0abdd49efea54add810ad0668df4a899', 'request_id': 'req-a4af7545-06fe-4d5d-9e09-1a6688ff077e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:46.529383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0abdd49efea54add810ad0668df4a899', 'request_id': 'req-a4af7545-06fe-4d5d-9e09-1a6688ff077e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:46.537747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0abdd49efea54add810ad0668df4a899', 'request_id': 'req-a4af7545-06fe-4d5d-9e09-1a6688ff077e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:46.548333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] The token cache is being invalidate because project 0abdd49efea54add810ad0668df4a899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 14:51:46.548625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:46.549276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4af7545-06fe-4d5d-9e09-1a6688ff077e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 14:51:46.551844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1214/2443] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 14:51:46 2022] DELETE /identity/v3/projects/0abdd49efea54add810ad0668df4a899 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:51:46.556766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:46.557252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:46.557362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:46.575614 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:46.576832 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:46.629424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:51:46.649674 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1230/2444] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:51:46.657063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:46.657469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:46.658036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:46.685494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:46.685494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:46.696589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1215/2445] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:51:46.700992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:46.713006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79c40f-6e27-4d55-acaf-cfbbf6cef747 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:46.717702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:46.718745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:46.719252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:46.719726 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:46.720882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 14:51:46.721438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:46.723906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:46.746168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1231/2446] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 14:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 14:51:46.747669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:46.769346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bcb7b-6909-4b12-abaa-ac2df06d6ac0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:46.779977 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:46.781078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:46.781649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:46.782107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:46.783204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:46.786721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:46.944179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1216/2447] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 14:51:46 2022] POST /identity/v3/projects => generated 389 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 14:51:46.944869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:46.958952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ba98bc-bb76-4686-99a0-4409359b00dd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:47.030110 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:47.030856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.031865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.032208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.033001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:47.035954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:47.139735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1232/2448] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 14:51:46 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 14:51:47.144050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:47.156556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f8cc1-9fa4-4be2-9227-36fcb1455ef4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:47.164460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:47.165171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.165453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.165728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.166460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:47.168518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:47.187704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1217/2449] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:51:47.190503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:47.201714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d4bce4-37ca-4108-a2cd-a392c6067cb7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:47.206871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:47.207712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.208070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.208412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.209214 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:47.211376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44df76c-f4eb-4a74-9c17-e707b746f2d6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:47.230267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1233/2450] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 14:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 14:51:47.234832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:47.249127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09fdf9-8811-4116-a74b-04d37ac51a0d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:47.255203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:47.256599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.256599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.257033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] PATH_INFO: `/v3/projects/cc0e507634064881b12a005738dbc30a/users/b5afc3a78f8e4ca68c97097cc51def69/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.275253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0e507634064881b12a005738dbc30a, user_id=b5afc3a78f8e4ca68c97097cc51def69, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:47.278197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:47.428534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1218/2451] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 14:51:47 2022] PUT /identity/v3/projects/cc0e507634064881b12a005738dbc30a/users/b5afc3a78f8e4ca68c97097cc51def69/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 14:51:47.433808 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb565ff-97f3-432c-a6d0-d22242daf727 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.434234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb565ff-97f3-432c-a6d0-d22242daf727 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.434532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb565ff-97f3-432c-a6d0-d22242daf727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.453049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aeb565ff-97f3-432c-a6d0-d22242daf727 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:47.454303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aeb565ff-97f3-432c-a6d0-d22242daf727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:47.519506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1234/2452] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 14:51:47.532576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:47.535543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:47.542427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:51:47.598065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb28a4-0690-41d2-b4d5-5fd343d5e2d2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:47.602480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:47.603158 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:47.603438 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:47.603712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:47.617190 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:47.618710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:47.628661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1219/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:51:53.547364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65466ef-9b79-4be3-815a-8d4413600b47 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:53.547910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65466ef-9b79-4be3-815a-8d4413600b47 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:53.547910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65466ef-9b79-4be3-815a-8d4413600b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:53.567796 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e65466ef-9b79-4be3-815a-8d4413600b47 None None] MFA Rules not processed for user `24999de5b283456fb3505d834c84133e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:53.570831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e65466ef-9b79-4be3-815a-8d4413600b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:53.624740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:51:53.634442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1235/2454] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:51:54.045604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:54.047793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:54.060309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8dc64-d9df-4ea4-b88f-7f506ed7c96f service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:54.064344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:54.065166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:54.065462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:54.065730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:54.068012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:54.079773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:54.081572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:54.093963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1220/2455] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:51:54.404516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b48e0-5f0e-4d98-8222-3297a9818c0f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:54.404806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b48e0-5f0e-4d98-8222-3297a9818c0f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:54.405088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b48e0-5f0e-4d98-8222-3297a9818c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:54.437399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-774b48e0-5f0e-4d98-8222-3297a9818c0f None None] MFA Rules not processed for user `b5afc3a78f8e4ca68c97097cc51def69`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:54.438836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-774b48e0-5f0e-4d98-8222-3297a9818c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:54.514563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1236/2456] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:51:54.529440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:51:54.530999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:54.637750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0fe77-53c6-4096-a5a4-b2783f474e35 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:51:54.644243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:51:54.644923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:54.645204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:54.645652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:54.654206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:51:54.655881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:51:54.667591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1221/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:51:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:51:57.880456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:51:57.880934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:51:57.881017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:51:57.900958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] MFA Rules not processed for user `b5afc3a78f8e4ca68c97097cc51def69`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:51:57.902197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:51:57.914824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1237/2458] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:51:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:52:04.663386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=64 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:52:07.894882 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=36 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:52:26.639274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:52:26.641356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:52:26.721044 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e4327-0758-4977-8e5e-0ed856245fcb service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:52:26.726274 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:52:26.726969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:52:26.727244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:52:26.727513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:52:26.729532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:52:26.743202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:52:26.744830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:52:26.756919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1222/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:52:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:52:36.752408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:54:04.087898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:54:04.089354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:54:04.145296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc89f4e-939d-4731-a97f-18946c172672 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:54:04.148920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:54:04.149573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:54:04.149852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:54:04.150123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:54:04.151916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:54:04.166159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:54:04.167755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:54:04.179187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1238/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:54:04.345945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:54:04.353806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72cfaeb-c6cc-49c4-8459-454abc9e3e9d service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:54:04.357530 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:54:04.358300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:54:04.358607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:54:04.358884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:54:04.360873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:54:04.376564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:54:04.379278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:54:04.389226 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1223/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:54:14.173838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=26 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:54:14.384306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:54:25.935340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:54:25.945298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958027eb-59d7-4b2e-9187-19f593659eff service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:54:25.949116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:54:25.950007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:54:25.950287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:54:25.950588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:54:25.952358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:54:25.962579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:54:25.964001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:54:25.971896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1239/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:54:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:54:35.968315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:55:06.154125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:55:06.155564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:55:06.167080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada0ed4-d84d-4006-9753-0c181d429e60 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:55:06.170690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:55:06.171325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:55:06.171629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:55:06.171902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:55:06.173631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:55:06.207276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:55:06.208703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:55:06.228018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1224/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:55:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:55:16.224357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=19 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:55:39.206426 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:55:39.250249 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28034099-d7aa-4fc1-a477-2f52012b858c service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:55:39.256912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:55:39.257564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:55:39.257852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:55:39.258127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:55:39.259945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:55:39.348403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:55:39.350004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:55:39.369126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1240/2464] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:55:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:55:45.087558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:55:45.106348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5893f6-dc24-4a00-8025-075466be8903 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:55:45.109947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:55:45.110628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:55:45.110906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:55:45.111178 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:55:45.113049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:55:45.192738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:55:45.194486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:55:45.212854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1225/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:55:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:55:49.366297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=59 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:55:55.210203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=46 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:56:55.268983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:56:55.297770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e5090-d768-4d53-b266-279c3e6144b7 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:56:55.301352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:56:55.301984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:56:55.302263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:56:55.302561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:56:55.310930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:56:55.312354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:56:55.321269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1241/2466] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:56:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:57:05.317319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=13 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:57:29.587424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:57:29.588867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:57:29.625034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11ed73-8a35-44ed-a28d-97291c9a1de1 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:57:29.630038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:57:29.630712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:57:29.630993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:57:29.631933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:57:29.640560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:57:29.642147 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:57:29.652312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1226/2467] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:57:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:57:39.648045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=15 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:30.278967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:30.297550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ad2ed-c1f8-4c8d-961d-b2325b2fb293 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:30.301128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:30.301756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:30.302036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:30.302307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:30.310776 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:30.312308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:30.321806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1242/2468] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:59:40.317881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 14:59:45.415392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:45.425169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909c125-69c9-46c5-b354-7c37ddb0ed4e service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:45.428994 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:45.429623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:45.429928 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:45.430196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:45.448499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:45.450435 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:45.472538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1227/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:59:47.597481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.599592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:47.628444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020646ea-a598-4aac-a68c-6f40c4316a66 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.633928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.634602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.634884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.642890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.642890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:47.650305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.651945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec3ade-6c22-4c25-a9d3-1bfbf6731c2b service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.663247 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1243/2470] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:59:47.870062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:47.880622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14dc2d-a880-4d76-8f31-bfe69d142d96 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:47.884336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:47.884974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:47.885444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:47.885721 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:47.898170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:47.905837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:47.911658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1228/2471] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:59:57.442653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:57.443165 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:57.443277 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:57.461248 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] MFA Rules not processed for user `b5afc3a78f8e4ca68c97097cc51def69`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 14:59:57.462449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:57.469884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1244/2472] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 14:59:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 14:59:57.782908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:57.791616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea2a646-dad3-4ce1-a74a-cde4cfac8876 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:57.795837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:57.796481 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:57.796786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:57.797064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:57.798853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:57.822019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:57.824855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:57.833001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1229/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 14:59:58.222711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 14:59:58.236046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999c96a-4146-4f20-9f52-089fba927d9e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 14:59:58.241290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 14:59:58.241947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 14:59:58.242231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 14:59:58.242509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 14:59:58.244371 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 14:59:58.257138 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 14:59:58.258748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 14:59:58.272830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1245/2474] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 14:59:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:00:05.693333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:05.703324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c72a5-012c-4ce8-ad47-10a786cc28b8 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:05.711427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:05.711427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:05.711427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:05.711427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:05.717848 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:05.719327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:05.729289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1230/2475] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:00:08.265080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=20 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:09.039933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:09.053997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04831-9dfe-4408-8ac6-730691f77ed3 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:09.061424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:09.062088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:09.062369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:09.062701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:09.064569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:00:09.079282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:09.080977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:09.093365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1246/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:00:15.724093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=36 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:19.088152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:40.243712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:40.252126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3bdb-92b2-44cd-949b-e33faf281270 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:40.255667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:40.256298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:40.256576 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:40.256846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:40.265770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:40.267214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:40.276781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1231/2477] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:00:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:00:45.624724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:00:45.634320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d8e477-2945-48de-95e1-3056ee8bac59 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:00:45.638119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:00:45.638794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:00:45.639079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:00:45.639394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:00:45.648533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:00:45.649946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:00:45.658775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1247/2478] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:00:50.271952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:00:55.654786 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:01:55.682769 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:01:55.691267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9a551-f560-4abd-b178-13c6e70fe42d service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:01:55.695170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:01:55.695801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:01:55.696080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:01:55.696351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:01:55.698156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:01:55.776386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:01:55.777838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:01:55.796561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1232/2479] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:01:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:02:05.793907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=41 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:02:34.445034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:02:34.446643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:34.517545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88dc43-11dd-4fcd-802a-51f4912b74db service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:02:34.523777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:02:34.524439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:02:34.524723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:02:34.525001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:02:34.527040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:02:34.540137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:02:34.541741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:02:34.553152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1248/2480] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:02:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:02:44.547378 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:35.253501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:35.255474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:35.311423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d456610-c2ab-4e18-8b2d-3dddf0aec4d7 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:35.314969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:35.315601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:35.315879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:35.316149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:35.318005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:35.327713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:35.329121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:35.337592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1233/2481] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:04:45.333391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=26 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:04:55.959961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:55.960843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:56.062074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742728f-c81e-4365-810c-85d1607a2aee service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:56.067831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:56.069417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:56.069417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:56.069417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:56.079687 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:56.081278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:56.092551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1249/2482] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:04:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:04:56.411315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:56.421189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20bdf-6fe9-441e-ad00-eb44ba48cf93 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:56.425649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:56.430352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:56.430352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:56.430352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:56.430352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:56.444594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:56.446038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:56.457177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1234/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:04:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:04:57.992423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:04:58.004206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cea3f-05b8-4f6f-9fd4-102a756c4f23 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:04:58.009780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:04:58.010444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:04:58.010770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:04:58.011082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:04:58.013485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:04:58.025762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:04:58.027491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:04:58.039073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1250/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:04:59.997209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:05:00.005929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f19260-44a4-48c3-bd14-928182032dc7 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:05:00.009539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:05:00.010175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:05:00.010472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:05:00.010772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:05:00.012630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:05:00.026421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:05:00.027873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:05:00.040216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1235/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:04:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:05:05.440333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:05:05.441913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:05:05.510400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb4bff-4302-4253-915f-ea81eb0af4ab service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:05:05.514116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:05:05.515618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:05:05.515901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:05:05.516177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:05:05.572865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:05:05.574306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:05:05.584845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1251/2486] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:05:10.034844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=12 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:05:15.579983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:05:40.436461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:05:40.445974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9db58-e6df-4ccc-b7ca-9542156411e8 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:05:40.450659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:05:40.451305 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:05:40.451583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:05:40.451861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:05:40.453705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:05:40.554854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:05:40.556409 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:05:40.574971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1236/2487] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:05:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:05:46.195406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:05:46.238239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e454b6fe-3f09-4fdf-b897-62b6276fcaa4 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:05:46.244439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:05:46.245230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:05:46.245512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:05:46.245788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:05:46.247980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:05:46.328136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:05:46.329762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:05:46.347768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1252/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:05:50.571776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=51 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:05:56.345126 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=55 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:06:58.210033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:06:58.211572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:06:58.252352 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1c5e-ec40-4b59-a8c0-b5f84f6dd605 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:06:58.256018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:06:58.256653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:06:58.256929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:06:58.257200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:06:58.265619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:06:58.267107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:06:58.276039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1237/2489] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:06:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:07:08.272085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=19 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:07:37.498368 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:07:37.499965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:07:37.535551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ad58-84e6-4f19-9f0b-df2b9c4f909a service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:07:37.540555 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:07:37.541194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:07:37.541472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:07:37.541742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:07:37.551003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:07:37.552564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:07:37.562267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1253/2490] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:07:47.558055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=15 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:09:57.485720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:09:57.504846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcffe3-0895-4e41-ac8c-9d4746139bf7 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:09:57.508652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:09:57.509280 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:09:57.509556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:09:57.509826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:09:57.527371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:09:57.528789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:09:57.549037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1238/2491] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:09:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:00.226085 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:00.235928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872624bf-34f7-4287-840e-0528792d4a89 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:00.239468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:00.240093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:00.240372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:00.240644 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:00.251439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:00.252848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:00.263977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1254/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:07.546810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=17 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:08.675340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:08.685106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795e1d3-b156-4b09-a914-43f84645ea77 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:08.688661 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:08.689295 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:08.689573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:08.689846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:08.691736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:08.725383 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:08.726851 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:08.746934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1239/2493] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:10.259774 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:18.743770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=19 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:41.403561 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:41.405182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:10:41.432734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdece5d6-5a5f-471a-b647-552554a42c07 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:41.438640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:41.439220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:41.439499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:41.439770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:41.448538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:41.450123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:41.459141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1255/2494] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:46.667480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:10:46.680347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf3ed6-4991-411d-9055-163110e47c48 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:10:46.685712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:10:46.686345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:10:46.686655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:10:46.686929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:10:46.695247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:10:46.696826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:10:46.706134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1240/2495] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:10:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:10:51.454881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=11 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:10:56.701855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:08.935673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:08.946698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21cfeb2-d113-4559-944a-797a2f7917f8 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:08.950394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:08.951081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:08.951365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:08.951650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:08.953576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:09.035716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:09.037195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4421ccf-7b79-4450-81ce-6fcebc8bcecb service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:09.057925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1256/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:09.343926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:09.354103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba8bbf-f068-4fd0-ae47-c92a018cccd5 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:09.357781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:09.358437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.358750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.359411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.361156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:09.454704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:09.456694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:09.476266 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1241/2497] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:09.512972 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:09.513268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:09.513548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:09.541534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] MFA Rules not processed for user `ea0f85ae8bd942338bfcda177b6864a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:09.542747 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:09.549988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1257/2498] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:12:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:10.024016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:10.032555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f8a0e-cce4-4c14-bf1b-61830c14895d service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:10.036789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:10.037431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:10.037716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:10.037995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:10.039897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:10.062156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:10.063641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:10.072524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1242/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:10.654664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:10.667469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede0b28-2a3c-42eb-8001-e7416b79b323 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:10.672438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:10.677712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:10.677712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:10.677712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:10.677712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:10.688422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:10.690179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:10.705592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1258/2500] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:13.481971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:13.495569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dbff1a-08f4-4de0-8ed7-98b85c93f9d7 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:13.500996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:13.501875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.502227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.503004 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.505163 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:13.517334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:13.519247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:13.529699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1243/2501] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:13.652299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:13.653751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:13.668123 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514bf0a-82e7-4a6e-bbd7-35604a33a2ec service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:13.672838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:13.673770 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.674723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.675000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] PATH_INFO: `/v3/users/24999de5b283456fb3505d834c84133e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.676362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24999de5b283456fb3505d834c84133e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:13.678300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:13.936063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:13.937670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:13.949961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517173d-a435-4a80-9932-2ca13a73d88e service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:13.953562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:13.954194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:13.954476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:13.954780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:13.962086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:13.963537 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:13.978715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1244/2502] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:12:13.988701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24999de5b283456fb3505d834c84133e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.005157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24999de5b283456fb3505d834c84133e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.073911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24999de5b283456fb3505d834c84133e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.082916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24999de5b283456fb3505d834c84133e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.083638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-16b375e0-a126-44af-8059-c94680ae9e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24999de5b283456fb3505d834c84133e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.100736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1259/2503] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:12:13 2022] DELETE /identity/v3/users/24999de5b283456fb3505d834c84133e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:12:14.456243 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:14.470262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca9bfa-fe86-4902-aa97-a3f7b3761caf service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:14.528934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:14.529797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.530083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.530363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] PATH_INFO: `/v3/projects/ac4f626daff54f49a6fee1d94fd991d8` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.531583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4f626daff54f49a6fee1d94fd991d8)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:14.534003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:14.553836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:14.676101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4f626daff54f49a6fee1d94fd991d8', 'request_id': 'req-1254a40a-001d-49b6-b70a-57b619d59e6a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.768624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4f626daff54f49a6fee1d94fd991d8', 'request_id': 'req-1254a40a-001d-49b6-b70a-57b619d59e6a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.775350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4f626daff54f49a6fee1d94fd991d8', 'request_id': 'req-1254a40a-001d-49b6-b70a-57b619d59e6a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.785946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] The token cache is being invalidate because project ac4f626daff54f49a6fee1d94fd991d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:12:14.786256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.786951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:12:14.788512 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1245/2504] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:12:14 2022] DELETE /identity/v3/projects/ac4f626daff54f49a6fee1d94fd991d8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:12:14.794929 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e61ba-7d88-4b45-91d8-f580122eda5b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.796939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e61ba-7d88-4b45-91d8-f580122eda5b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.796939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e61ba-7d88-4b45-91d8-f580122eda5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.816330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:14.864022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f98e61ba-7d88-4b45-91d8-f580122eda5b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:14.867142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f98e61ba-7d88-4b45-91d8-f580122eda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:14.875389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:14.878663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:14.946809 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1260/2505] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:14.955470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.955782 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.956179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.969755 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254a40a-001d-49b6-b70a-57b619d59e6a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:14.976982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:14.977931 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:14.978456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:14.978945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:14.986172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:14.986321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:14.987092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:14.994080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1261/2506] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:12:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:12:14.998273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.012502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c844f-eb7a-4263-89ae-a1305cffb485 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.014795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.014795 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.019563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.020846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.021301 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.021878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.022660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:12:15.022980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.024766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.028749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1246/2507] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:12:15.042576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1262/2508] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:12:15.045041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.057504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db18ef1-5bd4-4ba7-9c41-5b31de2e8567 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.063485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.064199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.064527 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.065550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.066332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.068741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.207510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1247/2509] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:12:15.211634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.222164 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae01afc-6beb-4c12-93f7-4f270170da4a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.282241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.282949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.283736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.284031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.284904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.288060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.448450 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1263/2510] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:12:15.455833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.466377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7d64-14da-438b-904e-5ab729539bdb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.471488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.472144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.472427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.472701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.473417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.475045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.484782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1248/2511] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:12:15.488754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.499329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d36ff-dae7-4ce3-ac34-4b9746ce5d58 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.504174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.515526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.515526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.515526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.515526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.515526 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4517ab39-20a8-4bae-a48d-18f53b480648 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.518435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1264/2512] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:12:15.522335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.532464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75da4e-6496-4c81-9a1b-342d205a167d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.537736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.538673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.538951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.539225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] PATH_INFO: `/v3/projects/a470bab157c74449bf906b5890caf162/users/4728df2cc48c4976a9f4be8eafcef6f9/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.554312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a470bab157c74449bf906b5890caf162, user_id=4728df2cc48c4976a9f4be8eafcef6f9, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.557049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.720472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1249/2513] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:12:15 2022] PUT /identity/v3/projects/a470bab157c74449bf906b5890caf162/users/4728df2cc48c4976a9f4be8eafcef6f9/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:12:15.733019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.733305 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.733607 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.761984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] MFA Rules not processed for user `4728df2cc48c4976a9f4be8eafcef6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:15.763321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:15.796268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:15.828012 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1265/2514] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:15.833040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:15.859452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:15.894224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28e102-f640-43f9-9166-7af4f4db09d6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:15.898923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:15.899572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:15.899860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:15.900130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:15.900821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:15.902820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:15.995857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1250/2515] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:12:15.999544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.011208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf150491-96e7-4b73-8374-6e60610a1fef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.068830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.069501 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.069818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.070095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.070836 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.072787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.181773 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1266/2516] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:12:15 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:12:16.185901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.196786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfad08c-4855-4ffe-a2f1-9494166a80f9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.201300 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.201954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.202234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.202538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.203256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.204850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.216357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1251/2517] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:12:16.220375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.230418 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe2b83-8786-499c-8f21-e61b44fbce9b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.235058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.235933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.236216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.237190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] PATH_INFO: `/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8/users/01cf4d93d73c4c649ea91a6259db896b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.252752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b98fd2eed614aa999e991fb96e3a3a8, user_id=01cf4d93d73c4c649ea91a6259db896b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.255459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.417172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1267/2518] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:12:16 2022] PUT /identity/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8/users/01cf4d93d73c4c649ea91a6259db896b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:12:16.421126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.432127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858655eb-ff9b-49ef-8e97-d080c7c9baa4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.490888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.491660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.492016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.492297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.493026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.494657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47f251-3186-462f-8f3f-0e353886f441 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.506081 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1252/2519] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:12:16.510044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:16.521338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2efa37-1c26-42a0-bd6f-689660f2695a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:16.526081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:16.526997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.527279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.527556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] PATH_INFO: `/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8/users/01cf4d93d73c4c649ea91a6259db896b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.529823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b98fd2eed614aa999e991fb96e3a3a8, user_id=01cf4d93d73c4c649ea91a6259db896b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:16.532705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:16.544340 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1268/2520] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:12:16 2022] PUT /identity/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8/users/01cf4d93d73c4c649ea91a6259db896b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:12:16.554728 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:16.555281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:16.555395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:16.582236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] MFA Rules not processed for user `01cf4d93d73c4c649ea91a6259db896b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:12:16.583446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:16.650879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1253/2521] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:12:20.281784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:20.353549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c80a2-5ce0-4743-8c76-df5ce6431bd6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:20.359177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:20.359829 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:20.360113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:20.360392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:20.362379 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:20.375269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:20.377580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:20.390877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1269/2522] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:24.924665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:24.926218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:24.946389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95e1a65-3cfc-4299-9441-80b7c93a10c8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:24.951904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:24.952552 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:24.952867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:24.953196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:24.955578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:24.968051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:24.969630 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:24.981802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1254/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:30.386074 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:34.977214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:43.558177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:43.559595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:43.629004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b1b3f-f0b8-46b4-8fcb-af3e42f3d318 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:43.633976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:43.634643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:43.634926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:43.635319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:43.637094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:43.648397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:43.649963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:43.660160 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1270/2524] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:12:53.655631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:12:58.697972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:12:58.699695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:12:58.756490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbecbb-a0c1-401a-b262-1b82b2ed6542 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:12:58.760159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:12:58.764390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:12:58.764390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:12:58.764390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:12:58.770591 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:12:58.772020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f456b390-90b7-4e7e-8d9f-965a7594bb41 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:12:58.781311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1255/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:12:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:13:08.777210 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=26 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:10.147217 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:10.220043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853849b-0032-480e-8b42-fcd494d42705 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:10.224771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:10.225666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:10.225995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:10.226233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] PATH_INFO: `/v3/users/4728df2cc48c4976a9f4be8eafcef6f9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:10.227453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4728df2cc48c4976a9f4be8eafcef6f9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:10.229378 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:10.444705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4728df2cc48c4976a9f4be8eafcef6f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:10.457159 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4728df2cc48c4976a9f4be8eafcef6f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:10.510394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4728df2cc48c4976a9f4be8eafcef6f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:10.518134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4728df2cc48c4976a9f4be8eafcef6f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:10.521219 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4728df2cc48c4976a9f4be8eafcef6f9'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:10.524779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1271/2526] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:14:10 2022] DELETE /identity/v3/users/4728df2cc48c4976a9f4be8eafcef6f9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:10.530111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:10.530627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:10.530740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:10.547947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:10.549136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:10.607085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1256/2527] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:10.617354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:10.618814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:10.675072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fa419-627f-4073-ba42-4e69903cc036 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:10.678682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:10.679316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:10.679598 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:10.679869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:10.688942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:10.690374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:10.700397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1272/2528] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:14:11.094846 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:11.097286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:11.108833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d8855-70ad-4efa-99d4-9b1f1935482d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:11.113674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:11.114589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:11.116110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:11.116396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] PATH_INFO: `/v3/projects/a470bab157c74449bf906b5890caf162` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:11.117938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a470bab157c74449bf906b5890caf162)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:11.119884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:11.258943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a470bab157c74449bf906b5890caf162', 'request_id': 'req-b824f7cb-f210-4ae7-8183-207ce7fc517c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.350565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a470bab157c74449bf906b5890caf162', 'request_id': 'req-b824f7cb-f210-4ae7-8183-207ce7fc517c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.357594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a470bab157c74449bf906b5890caf162', 'request_id': 'req-b824f7cb-f210-4ae7-8183-207ce7fc517c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.370198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] The token cache is being invalidate because project a470bab157c74449bf906b5890caf162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:14:11.370489 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.371182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.372733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1257/2529] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:14:11 2022] DELETE /identity/v3/projects/a470bab157c74449bf906b5890caf162 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:11.376761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:11.387035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece3ba6-ecba-44ab-b8ca-be00f517d6f7 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:11.405761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:11.443041 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:11.443941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:11.444235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:11.444496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] PATH_INFO: `/v3/users/01cf4d93d73c4c649ea91a6259db896b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:11.445712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01cf4d93d73c4c649ea91a6259db896b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:11.447659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:11.638835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01cf4d93d73c4c649ea91a6259db896b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.651078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01cf4d93d73c4c649ea91a6259db896b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.697917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01cf4d93d73c4c649ea91a6259db896b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.706205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01cf4d93d73c4c649ea91a6259db896b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.706845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d2424495-8e84-4cf7-8c40-4d41e198ca67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01cf4d93d73c4c649ea91a6259db896b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:11.712827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1273/2530] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:14:11 2022] DELETE /identity/v3/users/01cf4d93d73c4c649ea91a6259db896b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:12.111881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.117614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:12.128116 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824f7cb-f210-4ae7-8183-207ce7fc517c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.188241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.189086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.189363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.189647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] PATH_INFO: `/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.190843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b98fd2eed614aa999e991fb96e3a3a8)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.192754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.283504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:12.299674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b98fd2eed614aa999e991fb96e3a3a8', 'request_id': 'req-a48df436-96e2-4ca1-a2a4-fff0101c446e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:12.401958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b98fd2eed614aa999e991fb96e3a3a8', 'request_id': 'req-a48df436-96e2-4ca1-a2a4-fff0101c446e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:12.409789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b98fd2eed614aa999e991fb96e3a3a8', 'request_id': 'req-a48df436-96e2-4ca1-a2a4-fff0101c446e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:12.420316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] The token cache is being invalidate because project 7b98fd2eed614aa999e991fb96e3a3a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:14:12.420606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:12.421235 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a48df436-96e2-4ca1-a2a4-fff0101c446e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:14:12.422863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1258/2531] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:14:12 2022] DELETE /identity/v3/projects/7b98fd2eed614aa999e991fb96e3a3a8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:12.427888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.428374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.428487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.446213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:12.447369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:12.508367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1274/2532] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:12.517059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.517343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.517619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.536698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:12.538014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:12.546215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1259/2533] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:12.550404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.560209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db631a-2349-4f24-9f51-4fabee744f9b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.564759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.567080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.567080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.567080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.567080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:14:12.567080 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.568662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.581075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1275/2534] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:14:12.585182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.595047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e1ff7-2675-40f5-9929-35e50eebf2ae None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.599582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.600223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.600504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.600778 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.601450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.603402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.701298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1260/2535] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:14:12 2022] POST /identity/v3/projects => generated 357 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:12.705189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.716190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e39e8-4ac5-43f6-b235-6692e84aaf19 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.741779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:12.772581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.773265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.773712 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.774319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.775096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.777016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.853509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1276/2536] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:14:12 2022] POST /identity/v3/users => generated 279 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:12.857842 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.868618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e24d7-b098-4268-a961-940c1217d428 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.873835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.874618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.874900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.875175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.875901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.877523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.889314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1261/2537] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:12.893318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.903349 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60500b5f-a716-42f8-95cf-8ed09f7a32d5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.907975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.908604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.908884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.909158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.909867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.911485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389777f-6382-420c-8dac-ab1cf1459b46 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:12.921581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1277/2538] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:12.925509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:12.936905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4497353-1908-406c-9cea-54b2db78eb99 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:12.941801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:12.942756 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:12.943041 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:12.943319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] PATH_INFO: `/v3/projects/e333ec21a1cf434285f12f29189344ea/users/3cc3965dde01483495aab781b64b9860/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:12.960321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e333ec21a1cf434285f12f29189344ea, user_id=3cc3965dde01483495aab781b64b9860, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:12.962989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.048001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1262/2539] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:14:12 2022] PUT /identity/v3/projects/e333ec21a1cf434285f12f29189344ea/users/3cc3965dde01483495aab781b64b9860/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:13.060572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.060868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.061175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.088613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] MFA Rules not processed for user `3cc3965dde01483495aab781b64b9860`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:13.089857 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:13.156439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1278/2540] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:13.161057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.222460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11273-c323-4325-818c-3bbc846e3622 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.227002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.227653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.227922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.228199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.228879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.231468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.360082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1263/2541] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:14:13 2022] POST /identity/v3/projects => generated 357 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:13.367890 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.379465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba847a-b19e-49da-956d-b55234c6adf0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.437112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.437784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.438060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.438360 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.439079 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.441031 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.561951 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1279/2542] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:14:13 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:14:13.566006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.577106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac3084-23a0-4b99-9f44-96f0f092a1a9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.581573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.582230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.582534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.582809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.583533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.585126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.595906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1264/2543] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:13.603057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.613114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b89d9-b41d-45b6-8dda-269172c915db admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.618114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.619018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.619297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.619608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] PATH_INFO: `/v3/projects/87b5c153431b4f0eba60340c81a2e2ef/users/a4d7c7dd03324c0e904a77b0cce6137c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.636105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b5c153431b4f0eba60340c81a2e2ef, user_id=a4d7c7dd03324c0e904a77b0cce6137c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.639559 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.751918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1280/2544] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:14:13 2022] PUT /identity/v3/projects/87b5c153431b4f0eba60340c81a2e2ef/users/a4d7c7dd03324c0e904a77b0cce6137c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:13.756095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.766956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac739e-cf5b-428f-ac05-bff43b05939d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.786680 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:13.823944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.824555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.825419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.825698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.826447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.828082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be62ce-c6be-4b61-b330-88681d4cc6bd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.838794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1265/2545] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:14:13.842389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:13.847886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:13.853161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f50eca-f178-4060-a136-2505e923cb68 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:13.858374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:13.859280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.859562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.859838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] PATH_INFO: `/v3/projects/87b5c153431b4f0eba60340c81a2e2ef/users/a4d7c7dd03324c0e904a77b0cce6137c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.861841 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b5c153431b4f0eba60340c81a2e2ef, user_id=a4d7c7dd03324c0e904a77b0cce6137c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:13.864529 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:13.876135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1281/2546] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:14:13 2022] PUT /identity/v3/projects/87b5c153431b4f0eba60340c81a2e2ef/users/a4d7c7dd03324c0e904a77b0cce6137c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:14:13.886064 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:13.886356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:13.886748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:13.913381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] MFA Rules not processed for user `a4d7c7dd03324c0e904a77b0cce6137c`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:14:13.914666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:13.984432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1266/2547] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:14:13 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:14:23.980285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=52 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:24.069821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:14:59.070663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:59.074118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:14:59.147569 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899621a-4caa-45c1-9ee3-fc056b3ae8ce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:59.154507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:59.154507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:59.154507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:59.154507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:59.162690 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:59.164115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:59.174209 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1282/2548] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:14:59.503980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:14:59.514106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe97a6-2f2c-4d6f-84dc-393e1ec20058 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:14:59.517770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:14:59.518405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:14:59.518715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:14:59.518990 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:14:59.577649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:14:59.579127 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:14:59.588075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1267/2549] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:15:09.169600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=26 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:15:09.583966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=28 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:15:41.107208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:15:41.116762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc26ed-49b0-4a9b-931e-ca134e675a82 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:15:41.120345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:15:41.120973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:15:41.121252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:15:41.121524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:15:41.123289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:15:41.213431 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:15:41.214916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:15:41.234331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1283/2550] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:15:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:15:46.183206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:15:46.192513 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7caacc-3aab-47b8-8dc1-cd8e52281df3 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:15:46.196155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:15:46.196817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:15:46.197099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:15:46.197374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:15:46.199156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:15:46.271726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:15:46.273165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:15:46.282143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1268/2551] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:15:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:15:51.231411 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=45 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:15:56.277967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=34 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:16:58.853784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:16:58.855250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:58.933833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabf447-115d-46b3-b876-c62d8f07881a service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:16:58.937441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:16:58.938078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:16:58.938358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:16:58.938676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:16:58.940437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:16:59.008593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:16:59.010026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:16:59.019699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1284/2552] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:02.442800 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:02.444336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:02.527692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276035aa-68c9-4897-9c22-85a63ee49668 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:02.532749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:02.533386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:02.533671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:02.533946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:02.536540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:02.548345 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:02.550022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:02.560208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1269/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:17:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:06.315578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:06.317681 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:06.334230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131aafd-0102-4481-a8b3-5db97c64b4ea service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:06.339510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:06.340589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:06.340831 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:06.341107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] PATH_INFO: `/v3/users/7ec7931fdf144b3ba5f16970e0e2b00a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:06.361130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec7931fdf144b3ba5f16970e0e2b00a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:06.366528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:06.553190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec7931fdf144b3ba5f16970e0e2b00a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:06.565094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec7931fdf144b3ba5f16970e0e2b00a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:06.568415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:06.658861 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec7931fdf144b3ba5f16970e0e2b00a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:06.666793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec7931fdf144b3ba5f16970e0e2b00a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:06.667434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec7931fdf144b3ba5f16970e0e2b00a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:06.674068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1285/2554] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:17:06 2022] DELETE /identity/v3/users/7ec7931fdf144b3ba5f16970e0e2b00a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:17:06.918148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:06.930156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c4caa-b6af-41b1-bbe9-5026cbfeec86 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:06.988167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:06.989199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:06.989478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:06.989750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] PATH_INFO: `/v3/projects/e3e500b730cb4fb6a2841ff2750af31a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:07.000105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e500b730cb4fb6a2841ff2750af31a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:07.002087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:07.100736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e500b730cb4fb6a2841ff2750af31a', 'request_id': 'req-e616e61e-cfab-4364-9cf2-060c710b3591'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.173436 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e500b730cb4fb6a2841ff2750af31a', 'request_id': 'req-e616e61e-cfab-4364-9cf2-060c710b3591'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.180976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e500b730cb4fb6a2841ff2750af31a', 'request_id': 'req-e616e61e-cfab-4364-9cf2-060c710b3591'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.187887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:07.191554 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] The token cache is being invalidate because project e3e500b730cb4fb6a2841ff2750af31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:17:07.200868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.200868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.200868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1270/2555] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:17:06 2022] DELETE /identity/v3/projects/e3e500b730cb4fb6a2841ff2750af31a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:17:07.201330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:07.201330 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:07.213509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae06e42-9a87-42ca-83fd-81a78ea806df admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:07.271985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:07.272859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:07.273143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:07.273422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] PATH_INFO: `/v3/users/141acad0eb384b1a81ed815dd25761a0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:07.287756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=141acad0eb384b1a81ed815dd25761a0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:07.289758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:07.483181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '141acad0eb384b1a81ed815dd25761a0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.495866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '141acad0eb384b1a81ed815dd25761a0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.562314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '141acad0eb384b1a81ed815dd25761a0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.571533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '141acad0eb384b1a81ed815dd25761a0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.572319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '141acad0eb384b1a81ed815dd25761a0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:07.578427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1286/2556] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:17:07 2022] DELETE /identity/v3/users/141acad0eb384b1a81ed815dd25761a0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:17:07.815262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:07.816741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:07.830113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616e61e-cfab-4364-9cf2-060c710b3591 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:07.890608 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:07.891415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:07.891693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:07.891964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] PATH_INFO: `/v3/projects/b82620357804469d824314f3de90901b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:07.903378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b82620357804469d824314f3de90901b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:07.905442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:08.019495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b82620357804469d824314f3de90901b', 'request_id': 'req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:08.079378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b82620357804469d824314f3de90901b', 'request_id': 'req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:08.086667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b82620357804469d824314f3de90901b', 'request_id': 'req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:08.097654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] The token cache is being invalidate because project b82620357804469d824314f3de90901b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:17:08.097939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:08.098618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:17:08.100834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1271/2557] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:17:07 2022] DELETE /identity/v3/projects/b82620357804469d824314f3de90901b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:17:08.601105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:08.602827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:08.660509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:08.686409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e7a3f-b971-40c9-a28a-0c60c49ed6df admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:08.737387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:08.738084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:08.738397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:08.738701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:08.745347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:08.799810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:08.801267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:08.810953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1287/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:17:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:18.094220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:18.807208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=53 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:17:47.464981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:47.468669 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:47.556546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1db86f-5cd8-496c-9a6c-974c7fb0edb1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:47.561555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:47.562136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:47.562413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:47.562716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:47.564446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:47.576229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:47.577792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:47.587719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1272/2559] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:50.080432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:17:50.081979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:50.141758 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc433d8-c899-49ab-8cc7-5416c2f6b143 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:17:50.145543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:17:50.146193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:17:50.146478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:17:50.146807 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:17:50.149507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:17:50.159719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:17:50.161156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:17:50.170267 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1288/2560] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:17:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:17:57.583542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=41 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:18:00.166045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=26 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:19:59.466786 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:19:59.468197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:59.480103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d09bd-90b6-44ce-9444-257f06f49050 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:19:59.483735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:19:59.484999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:19:59.484999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:19:59.484999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:19:59.486735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:19:59.546966 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:19:59.548445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:19:59.557462 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1273/2561] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:09.553420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=28 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:41.368589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:41.378070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c78fd2-6ebd-4bd7-8aeb-2f793f84db14 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:41.381634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:41.382286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:41.382592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:41.382868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:41.384593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:41.444382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:41.445879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:41.454838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1289/2562] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:51.450723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=28 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:20:58.664427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:58.665830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:58.688547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d3d5d-36a7-49bf-acbc-396388ef27ac service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:58.692956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:58.693607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:58.694745 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:58.695066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:58.697080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:58.790970 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:58.792419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:58.813131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1274/2563] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:59.657434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:59.666847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea61ddd-c8c0-40c0-910b-342739535269 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:20:59.670469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:20:59.671108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:20:59.671386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:20:59.671656 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:20:59.673436 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:20:59.741695 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:20:59.743228 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:20:59.753339 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1290/2564] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:20:59.987323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:20:59.997361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2f3f8-aa17-4e83-8c76-a35a7b70230a service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:00.001172 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:00.001889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:00.002177 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:00.002457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:00.004858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:00.078735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:00.078735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:00.078735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1275/2565] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:20:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:00.307351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:00.309037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:00.329197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb415d9-a4b4-45d0-836f-78e5ddbce4cc service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:00.335918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:00.336663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:00.337009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:00.337348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:00.339335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:00.355933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:00.358343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:00.375342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1291/2566] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:03.654431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:03.656144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:03.675431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a154d7-4923-4108-911b-575f5de5d2ca service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:03.680306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:03.681216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:03.681502 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:03.681781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] PATH_INFO: `/v3/users/b5afc3a78f8e4ca68c97097cc51def69` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:03.683020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5afc3a78f8e4ca68c97097cc51def69)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:03.685128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:03.900336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5afc3a78f8e4ca68c97097cc51def69'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:03.912253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5afc3a78f8e4ca68c97097cc51def69'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:03.988871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5afc3a78f8e4ca68c97097cc51def69'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:03.996972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5afc3a78f8e4ca68c97097cc51def69'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:03.997651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c152181b-757b-41c8-a7b5-efaebe74584c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5afc3a78f8e4ca68c97097cc51def69'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.003704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1276/2567] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:03 2022] DELETE /identity/v3/users/b5afc3a78f8e4ca68c97097cc51def69 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:04.317772 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:04.326201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c3d00-838b-4e5d-bd8e-2a48b5d9cddf service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:04.387390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:04.388179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.388456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.388727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] PATH_INFO: `/v3/projects/cc0e507634064881b12a005738dbc30a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.389897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0e507634064881b12a005738dbc30a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:04.391827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:04.496887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0e507634064881b12a005738dbc30a', 'request_id': 'req-7fc31b99-ee01-45bb-94e5-88c462768f9b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.503056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0e507634064881b12a005738dbc30a', 'request_id': 'req-7fc31b99-ee01-45bb-94e5-88c462768f9b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.576086 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0e507634064881b12a005738dbc30a', 'request_id': 'req-7fc31b99-ee01-45bb-94e5-88c462768f9b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.587689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] The token cache is being invalidate because project cc0e507634064881b12a005738dbc30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:04.588116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.588739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7fc31b99-ee01-45bb-94e5-88c462768f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:04.590315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1292/2568] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:04 2022] DELETE /identity/v3/projects/cc0e507634064881b12a005738dbc30a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:04.596678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.596967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.597246 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.616674 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:04.617884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:04.622340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:04.677396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1277/2569] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:04.685452 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.685743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.686019 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.704161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:04.705365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:04.713902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1293/2570] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:04.718251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:04.728886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663c8088-1444-4514-b23f-1524bd70c4f0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:04.735262 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:04.735980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.736258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.736534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.737264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:04.737571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:04.739241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:04.751762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1278/2571] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:04.755907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:04.762392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:04.768964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b2e26-1b0e-468c-8561-b1f652c86956 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:04.773733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:04.774392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.775365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.775638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.776428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:04.778405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:04.888862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1294/2572] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:04 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:04.892992 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:04.903684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f8c2-ff61-424e-a038-c59816c8afe4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:04.961117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:04.961708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:04.961991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:04.962962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:04.963657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:04.965639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.057416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1279/2573] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:04 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:05.061516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.072487 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75028d01-9286-4837-9d60-3ba7116ae7de admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.077253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.086551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.086551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.086551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.086551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.086551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.095532 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1295/2574] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:05.099453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.110074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ebda2-3c6d-41e4-8d45-c75b0406f3e5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.114942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.115604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.115890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.116166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.116891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.119023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484be72-502e-428e-aabb-7b2ac72e2173 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.129598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1280/2575] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:05.133566 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.144240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c1718-46f9-4fa7-8a9e-7062ed986640 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.149270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.149905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.150184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.150464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] PATH_INFO: `/v3/projects/7ec6e362de2645e99ec4cb5dfcbe9bc3/users/852ef3864a4a4bf096c9ddf1ac283bac/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.165764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec6e362de2645e99ec4cb5dfcbe9bc3, user_id=852ef3864a4a4bf096c9ddf1ac283bac, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.168382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.242106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1296/2576] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:05 2022] PUT /identity/v3/projects/7ec6e362de2645e99ec4cb5dfcbe9bc3/users/852ef3864a4a4bf096c9ddf1ac283bac/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:21:05.253256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.253618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.253901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.281758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] MFA Rules not processed for user `852ef3864a4a4bf096c9ddf1ac283bac`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:05.283018 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:05.347498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1281/2577] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:05.352020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.431642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d6999-dd80-4aa1-bc7f-f933083d10ce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.436942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.437631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.437918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.438823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.439544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.442624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.560535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1297/2578] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:05 2022] POST /identity/v3/projects => generated 369 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:05.564647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.571490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:05.575862 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bf37d-c3a8-41ad-b06a-61dfe8ec9114 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.633016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.633688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.634249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.634845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.635936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.637896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.738016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1282/2579] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:05 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:05.741729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.752813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b0e7b-0f85-46e5-8501-78b8c9e5b60b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.757592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.758279 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.758590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.758926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.759651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.761285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.772370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1298/2580] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:05.780833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.791333 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4010f59-833e-4957-bf94-315ebae6757c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.796166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.797146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.797428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.797702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] PATH_INFO: `/v3/projects/62ad0995ee144b008c59b01bf0354feb/users/ac827ea5fb4844a4ac3f90b7e25d009e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.813753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ad0995ee144b008c59b01bf0354feb, user_id=ac827ea5fb4844a4ac3f90b7e25d009e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.816540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.897816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1283/2581] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:05 2022] PUT /identity/v3/projects/62ad0995ee144b008c59b01bf0354feb/users/ac827ea5fb4844a4ac3f90b7e25d009e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:05.902324 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.912562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f080a3-eb0f-45ca-8a83-edc7cc188132 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:05.967974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:05.968645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:05.968930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:05.969319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:05.970043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:05.971673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afeb78-9370-4c08-9b1e-9414d63c5c68 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:05.982162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1299/2582] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:05.986504 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:05.997068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345941-a847-49ab-a3fc-486dcf33a137 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:06.001633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:06.002538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:06.003332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:06.003332 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] PATH_INFO: `/v3/projects/62ad0995ee144b008c59b01bf0354feb/users/ac827ea5fb4844a4ac3f90b7e25d009e/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:06.005223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ad0995ee144b008c59b01bf0354feb, user_id=ac827ea5fb4844a4ac3f90b7e25d009e, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:06.007916 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:06.018233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1284/2583] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:05 2022] PUT /identity/v3/projects/62ad0995ee144b008c59b01bf0354feb/users/ac827ea5fb4844a4ac3f90b7e25d009e/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:06.030168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:06.030465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:06.030802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:06.058905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] MFA Rules not processed for user `ac827ea5fb4844a4ac3f90b7e25d009e`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:06.060128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:06.065129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:06.128713 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1300/2584] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:06.139565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:06.141250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:06.198531 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704f1c1-a573-4874-8fad-b158656a0c74 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:06.202113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:06.202772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:06.203051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:06.203324 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:06.212100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:06.214222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d4598-8c6a-46c1-8ca1-e2bf80f96b9c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:06.223511 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1285/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:09.320432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.384272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad4fc5-33e8-4c78-9992-05669d732184 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.389608 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.390462 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.390793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.391063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] PATH_INFO: `/v3/users/852ef3864a4a4bf096c9ddf1ac283bac` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.392240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=852ef3864a4a4bf096c9ddf1ac283bac)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.394180 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.660639 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852ef3864a4a4bf096c9ddf1ac283bac'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:09.673454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852ef3864a4a4bf096c9ddf1ac283bac'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:09.761631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852ef3864a4a4bf096c9ddf1ac283bac'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:09.769580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852ef3864a4a4bf096c9ddf1ac283bac'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:09.770168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852ef3864a4a4bf096c9ddf1ac283bac'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:09.777106 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1301/2586] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:09 2022] DELETE /identity/v3/users/852ef3864a4a4bf096c9ddf1ac283bac => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:09.782117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.782656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.782878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.800716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:09.801926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:09.856410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:09.864817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1286/2587] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:21:09.875099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:09.876504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:09.903965 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:09.933912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2801b-fdc1-4963-845e-6a2e428c9e1f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:09.937883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:09.938556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:09.938843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:09.939119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:09.953994 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:09.955497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:09.967399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1302/2588] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:10.437449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:10.438867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:10.456086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb2669-4c1d-4d16-914a-4622eec99c0f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:10.458222 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:10.459250 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:10.459529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:10.460419 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] PATH_INFO: `/v3/projects/7ec6e362de2645e99ec4cb5dfcbe9bc3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:10.461779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec6e362de2645e99ec4cb5dfcbe9bc3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:10.464716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:10.675424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec6e362de2645e99ec4cb5dfcbe9bc3', 'request_id': 'req-abcb0854-96de-4f70-85f4-b33bed3f570c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:10.784533 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec6e362de2645e99ec4cb5dfcbe9bc3', 'request_id': 'req-abcb0854-96de-4f70-85f4-b33bed3f570c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:10.791429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec6e362de2645e99ec4cb5dfcbe9bc3', 'request_id': 'req-abcb0854-96de-4f70-85f4-b33bed3f570c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:10.802057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] The token cache is being invalidate because project 7ec6e362de2645e99ec4cb5dfcbe9bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:10.802348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:10.802998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:10.804604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1287/2589] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:10 2022] DELETE /identity/v3/projects/7ec6e362de2645e99ec4cb5dfcbe9bc3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:10.808686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:10.818280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2596b-0a65-46d2-ad57-86ecba0b3dca service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:10.881154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:10.882021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:10.882297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:10.882595 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] PATH_INFO: `/v3/users/ac827ea5fb4844a4ac3f90b7e25d009e` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:10.883788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac827ea5fb4844a4ac3f90b7e25d009e)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:10.886287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:11.113434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac827ea5fb4844a4ac3f90b7e25d009e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.125877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac827ea5fb4844a4ac3f90b7e25d009e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.172547 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac827ea5fb4844a4ac3f90b7e25d009e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.179819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac827ea5fb4844a4ac3f90b7e25d009e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.180448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0eaa89fe-4817-4972-9e40-0cf79168417b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac827ea5fb4844a4ac3f90b7e25d009e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.186429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1303/2590] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:21:10 2022] DELETE /identity/v3/users/ac827ea5fb4844a4ac3f90b7e25d009e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:11.638632 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:11.645809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:11.655239 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb0854-96de-4f70-85f4-b33bed3f570c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:11.715027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:11.715895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:11.716176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:11.716450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] PATH_INFO: `/v3/projects/62ad0995ee144b008c59b01bf0354feb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:11.717637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ad0995ee144b008c59b01bf0354feb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:11.719601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:11.887156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ad0995ee144b008c59b01bf0354feb', 'request_id': 'req-2e3b8141-a42a-410a-bd04-0e277695c431'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.955754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ad0995ee144b008c59b01bf0354feb', 'request_id': 'req-2e3b8141-a42a-410a-bd04-0e277695c431'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.962638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ad0995ee144b008c59b01bf0354feb', 'request_id': 'req-2e3b8141-a42a-410a-bd04-0e277695c431'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.973202 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] The token cache is being invalidate because project 62ad0995ee144b008c59b01bf0354feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:21:11.973485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.974133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2e3b8141-a42a-410a-bd04-0e277695c431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:21:11.975703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1288/2591] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:21:11 2022] DELETE /identity/v3/projects/62ad0995ee144b008c59b01bf0354feb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:11.980655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:11.981135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:11.981241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:11.999338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:12.000535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:12.041682 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:12.061199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1304/2592] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:12.068955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.069244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.069519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.091194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:12.092460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:12.105183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1289/2593] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:12.105632 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.115676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284af15-e546-430c-91bf-16ced3cd511d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.122070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.122843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.123130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.123524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.124327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:21:12.124659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.126310 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.139660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1305/2594] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:21:12.143858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.153822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2fed7-a18f-4221-83fd-cd5ba50c5433 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.158493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.159165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.159443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.159716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.160396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.162285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.165474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:12.249017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1290/2595] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:12.253282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.264346 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb048eac-6b65-43ca-b394-18e0384e4799 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.321581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.322252 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.322600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.323446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.324147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.326094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.403538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1306/2596] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:12.407732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.418716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dc9c1-ccc2-4215-a84b-1a36edf33211 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.423607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.424264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.424580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.424858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.425586 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.427223 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.437837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1291/2597] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:12.446089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.456666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c1bc-310f-4c73-b4a0-4801991b87f7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.461328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.461980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.462263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.462562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.463294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.465464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718a13-2a4b-4832-a29a-ec77a2495672 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.475412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1307/2598] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:12.487179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.497232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24383981-a899-43da-b2ea-72ac3abdc619 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.501816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.506900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.506900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.506900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] PATH_INFO: `/v3/projects/188bc6aca6424e6196cd504ca6576c46/users/95d38df986a44421ac495b185658c13a/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.518414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] RBAC: Authorizing `identity:create_grant(project_id=188bc6aca6424e6196cd504ca6576c46, user_id=95d38df986a44421ac495b185658c13a, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.521302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.614171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1292/2599] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:12 2022] PUT /identity/v3/projects/188bc6aca6424e6196cd504ca6576c46/users/95d38df986a44421ac495b185658c13a/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:12.624971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692197-118e-404c-bb39-eac43c784766 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.625265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692197-118e-404c-bb39-eac43c784766 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.625551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692197-118e-404c-bb39-eac43c784766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.654243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ea692197-118e-404c-bb39-eac43c784766 None None] MFA Rules not processed for user `95d38df986a44421ac495b185658c13a`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:12.655428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea692197-118e-404c-bb39-eac43c784766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:12.721135 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1308/2600] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:12.725581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.789747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59d07f-f720-4de7-8d95-b01536c2ffde admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.794709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:12.795360 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:12.795638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:12.795912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:12.796597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:12.798488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:12.929143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1293/2601] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:21:12.934251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692197-118e-404c-bb39-eac43c784766 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:12.945014 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692197-118e-404c-bb39-eac43c784766 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:12.987212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:13.005514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.006403 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.006549 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.006881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.007606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.009515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.202729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1309/2602] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:21:12 2022] POST /identity/v3/users => generated 298 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:21:13.206452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.217392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a2319-43a3-4ec0-a46c-ee7aa68171da admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.221910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.222594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.222879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.223156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.223878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.225469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.236717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1294/2603] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:13.244194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.254414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e995d-7679-4fa3-81b2-7ca0dd2ef95e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.259133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.260006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.260285 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.261058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] PATH_INFO: `/v3/projects/77a115516c5245a9b943b9f6fd8f0764/users/766cb439dcc84c61a5369e0f0227b58b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.275794 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a115516c5245a9b943b9f6fd8f0764, user_id=766cb439dcc84c61a5369e0f0227b58b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.279161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.374793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1310/2604] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:13 2022] PUT /identity/v3/projects/77a115516c5245a9b943b9f6fd8f0764/users/766cb439dcc84c61a5369e0f0227b58b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:13.378921 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.390078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d2977-04b1-4e27-89e9-d6b2e8196899 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.444662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.449713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.449713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.449713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.449713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.449713 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb77d6-9284-4e70-a06b-2fb92a3601c6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.458658 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1295/2605] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:21:13.462785 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.472963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a1232-b439-4f72-9cc9-6131cf5068b8 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.479025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.479025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.479025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.479025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] PATH_INFO: `/v3/projects/77a115516c5245a9b943b9f6fd8f0764/users/766cb439dcc84c61a5369e0f0227b58b/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.481140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a115516c5245a9b943b9f6fd8f0764, user_id=766cb439dcc84c61a5369e0f0227b58b, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.483821 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.494659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1311/2606] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:21:13 2022] PUT /identity/v3/projects/77a115516c5245a9b943b9f6fd8f0764/users/766cb439dcc84c61a5369e0f0227b58b/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:21:13.506613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.506999 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.507284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.534316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] MFA Rules not processed for user `766cb439dcc84c61a5369e0f0227b58b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:21:13.535548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:13.544249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:13.605932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1296/2607] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:21:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:21:13.617094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.618863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:21:13.677813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a870035-24f1-4cd4-9dac-cfff4d558b81 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.681650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.682823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.683090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.683449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.692545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.693973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.704059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1312/2608] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:13.800253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:21:13.810421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce41dcc-3ae3-488a-b771-f4a66204e8eb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:21:13.814052 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:21:13.814723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:21:13.815005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:21:13.815279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:21:13.871456 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:21:13.872893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:21:13.881752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1297/2609] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:21:23.700199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=52,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:21:23.877806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=53 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:22:54.545985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:22:54.548199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:54.617880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a763cd9-1e18-4d92-80c0-9c525b6128e5 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:22:54.622988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:22:54.623578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:22:54.623863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:22:54.624141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:22:54.627005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:22:54.638611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:22:54.640221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:22:54.651235 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1313/2610] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:23:04.646935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:08.309411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:08.310859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:08.325072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6180e9-56a1-49d7-a099-0e359cf17efb service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:08.328768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:08.329408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:08.329692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:08.329969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:08.331761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:08.397730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:08.399335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391a5e3-8dd1-4de2-84db-ab1958845619 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:08.409299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1298/2611] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:08 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:09.075580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:09.168186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee08ae6a-09ec-4910-9ecd-afa06f51e6a0 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:09.172787 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:09.173659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:09.173936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:09.174208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] PATH_INFO: `/v3/users/95d38df986a44421ac495b185658c13a` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:09.175407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d38df986a44421ac495b185658c13a)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:09.177314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:09.383944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d38df986a44421ac495b185658c13a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:09.396212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d38df986a44421ac495b185658c13a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:09.465907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d38df986a44421ac495b185658c13a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:09.473680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d38df986a44421ac495b185658c13a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:09.474327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d38df986a44421ac495b185658c13a'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:09.480356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1314/2612] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:09 2022] DELETE /identity/v3/users/95d38df986a44421ac495b185658c13a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:09.484939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:09.485424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:09.485515 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:09.526896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:09.528165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:09.585989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1299/2613] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:09.597148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:09.598576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:09.666957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbad3b6-8286-4ee2-b20b-48aef238f1a0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:09.670556 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:09.671194 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:09.671477 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:09.671750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:09.677605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:09.682033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:09.683517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:09.693104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1315/2614] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:10.130259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:10.132395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:10.144655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02bc04-5258-4ebe-a9d4-9fd191a25725 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:10.151286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:10.152558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:10.152840 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:10.153159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] PATH_INFO: `/v3/projects/188bc6aca6424e6196cd504ca6576c46` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:10.154485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=188bc6aca6424e6196cd504ca6576c46)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:10.157717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:10.249118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '188bc6aca6424e6196cd504ca6576c46', 'request_id': 'req-b1b5d728-21ce-49a4-804b-2b387816e3dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.334279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '188bc6aca6424e6196cd504ca6576c46', 'request_id': 'req-b1b5d728-21ce-49a4-804b-2b387816e3dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.342675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '188bc6aca6424e6196cd504ca6576c46', 'request_id': 'req-b1b5d728-21ce-49a4-804b-2b387816e3dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.350080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:10.354285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] The token cache is being invalidate because project 188bc6aca6424e6196cd504ca6576c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:10.354620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.355276 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.357048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1300/2615] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:10 2022] DELETE /identity/v3/projects/188bc6aca6424e6196cd504ca6576c46 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:10.361037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:10.375162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a855928-ef01-4039-9848-d129e8c9fdf2 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:10.437063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:10.437947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:10.438229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:10.438504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] PATH_INFO: `/v3/users/766cb439dcc84c61a5369e0f0227b58b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:10.439748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=766cb439dcc84c61a5369e0f0227b58b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:10.441666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:10.621322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766cb439dcc84c61a5369e0f0227b58b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.633996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766cb439dcc84c61a5369e0f0227b58b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.688604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766cb439dcc84c61a5369e0f0227b58b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.696953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766cb439dcc84c61a5369e0f0227b58b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.697604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3ec80501-c23b-40fb-ae06-0f34c416ac6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766cb439dcc84c61a5369e0f0227b58b'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:10.703760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1316/2616] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:10 2022] DELETE /identity/v3/users/766cb439dcc84c61a5369e0f0227b58b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:11.176334 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:11.177863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:11.194747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5d728-21ce-49a4-804b-2b387816e3dd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:11.256386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:11.257275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.257708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.258428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] PATH_INFO: `/v3/projects/77a115516c5245a9b943b9f6fd8f0764` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.259687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a115516c5245a9b943b9f6fd8f0764)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:11.261686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:11.386068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a115516c5245a9b943b9f6fd8f0764', 'request_id': 'req-d23acef1-24ea-4730-a443-dbee90639367'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:11.494666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a115516c5245a9b943b9f6fd8f0764', 'request_id': 'req-d23acef1-24ea-4730-a443-dbee90639367'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:11.501487 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a115516c5245a9b943b9f6fd8f0764', 'request_id': 'req-d23acef1-24ea-4730-a443-dbee90639367'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:11.512963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] The token cache is being invalidate because project 77a115516c5245a9b943b9f6fd8f0764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:11.513247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:11.513898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d23acef1-24ea-4730-a443-dbee90639367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:11.515491 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1301/2617] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:11 2022] DELETE /identity/v3/projects/77a115516c5245a9b943b9f6fd8f0764 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:11.520322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.520791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.520894 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.568612 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:11.569817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:11.614458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:11.629755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1317/2618] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:11.638619 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.638911 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.639196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.658907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:11.660130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:11.668261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1302/2619] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:11.672372 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:11.682328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15404b4e-98e2-4ad8-bec9-a635848bb49e None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:11.687584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:11.688256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.688539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.688811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.689531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:11.689833 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:11.691455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:11.703620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1318/2620] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:11.707768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:11.717672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd2931-d1d2-4c78-bf2c-1e41a08da56e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:11.722218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:11.722892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.723170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.723441 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.724124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:11.726013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:11.844565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1303/2621] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:11 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:11.848803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:11.859927 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddcba-667c-4f5d-a59d-e3c8c8e47f37 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:11.917860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:11.918545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:11.918850 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:11.919147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:11.919839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:11.921757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.024483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1319/2622] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:11 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:12.025730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.036329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7cc75-f616-4102-a73d-a93f774b72e5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.041969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.042711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.042993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.043278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.044009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.045671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.059470 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1304/2623] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:12.060928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.080474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6fcf2-6c4b-4d6f-b477-6d4fd8d7da91 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.082697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.083367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.083669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.083933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.084689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.086940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0831eca0-149e-4a4f-9c75-8b90dea4c011 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.098141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1320/2624] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:12.102891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.113281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd5e04-1d4a-432b-be81-f6bcab5b465f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.118047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.118983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.119268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.119539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] PATH_INFO: `/v3/projects/8f80c3328f534e70b9079cf1f1a6b2b3/users/4737baee043d45bbaef04ca2ebb4e38f/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.135961 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f80c3328f534e70b9079cf1f1a6b2b3, user_id=4737baee043d45bbaef04ca2ebb4e38f, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.140188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.227828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1305/2625] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:12 2022] PUT /identity/v3/projects/8f80c3328f534e70b9079cf1f1a6b2b3/users/4737baee043d45bbaef04ca2ebb4e38f/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:12.238099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.238389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.238701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.266290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] MFA Rules not processed for user `4737baee043d45bbaef04ca2ebb4e38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:12.267550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:12.330234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1321/2626] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:12.334850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.356565 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:12.400119 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de64c1f-9e55-4516-9cbc-fe5d30424f58 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.405121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.405785 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.406066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.406385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.407115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.409101 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.514375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1306/2627] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:12 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:12.518211 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.529665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6a9c9-ddd3-4c28-b20d-a0c3ebb5393c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.572916 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:12.588202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.588866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.589146 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.589421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.590094 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.592060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.678906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1322/2628] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:12 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:12.687291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.699346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a2e9e-3c3d-4e9b-a350-0a0a004c49cb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.704534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.705212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.705499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.705775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.707187 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.708805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.720303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1307/2629] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:12.724562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.735388 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c0c68-da26-4478-a381-397bb2a6dc6b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.740447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.741401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.741648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.742006 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] PATH_INFO: `/v3/projects/f6d15fca26f34671a1a043978258bece/users/e18bbde04b7342ac895cfa8520a2d3c6/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.761668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d15fca26f34671a1a043978258bece, user_id=e18bbde04b7342ac895cfa8520a2d3c6, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.764655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.849669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1323/2630] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:12 2022] PUT /identity/v3/projects/f6d15fca26f34671a1a043978258bece/users/e18bbde04b7342ac895cfa8520a2d3c6/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:12.854154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.864059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560787b-90fe-441d-a93c-8d7ed1c2fecc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.923898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.924582 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.924887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.925163 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.925891 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.927665 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1968e17-f145-4906-b9df-b8e7327e1d7d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.938654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1308/2631] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:12.942760 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:12.953864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346790a-52e6-45ed-a868-cc412a17eb99 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:12.958606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:12.959478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.959761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.960037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] PATH_INFO: `/v3/projects/f6d15fca26f34671a1a043978258bece/users/e18bbde04b7342ac895cfa8520a2d3c6/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:12.962114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d15fca26f34671a1a043978258bece, user_id=e18bbde04b7342ac895cfa8520a2d3c6, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:12.964779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:12.975430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1324/2632] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:12 2022] PUT /identity/v3/projects/f6d15fca26f34671a1a043978258bece/users/e18bbde04b7342ac895cfa8520a2d3c6/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:12.986725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:12.987017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:12.987289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:13.014859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] MFA Rules not processed for user `e18bbde04b7342ac895cfa8520a2d3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:13.016088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:13.085548 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1309/2633] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:13.097843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:13.099590 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:13.158875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30523154-600b-4e4f-b0fd-0843954ac25b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:13.162577 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:13.163220 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:13.163502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:13.163777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:13.220318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:13.221668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0eded6-44a2-46b1-8078-4974c60c8a0a service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:13.231539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1325/2634] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:17.333572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:17.347892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:17.398444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f093d3bf-5330-4a7d-8477-a32613b07b15 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:17.404774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:17.405703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:17.405991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:17.406270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] PATH_INFO: `/v3/users/4737baee043d45bbaef04ca2ebb4e38f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:17.407553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4737baee043d45bbaef04ca2ebb4e38f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:17.409571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:17.606028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4737baee043d45bbaef04ca2ebb4e38f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:17.618510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4737baee043d45bbaef04ca2ebb4e38f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:17.695258 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4737baee043d45bbaef04ca2ebb4e38f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:17.703561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4737baee043d45bbaef04ca2ebb4e38f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:17.704198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4737baee043d45bbaef04ca2ebb4e38f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:17.710086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1310/2635] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:17 2022] DELETE /identity/v3/users/4737baee043d45bbaef04ca2ebb4e38f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:17.715204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:17.715676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:17.715764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:17.732528 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:17.733789 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:17.778819 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:17.792662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1326/2636] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:17.802236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:17.807886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:17.866159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474642fa-a5de-49b8-ac0b-5730866f3a7f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:17.870424 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:17.871107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:17.871405 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:17.871677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:17.880957 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:17.882420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:17.893254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1311/2637] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:18.326130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:18.327631 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:18.340613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934af4c8-2564-4b98-9ecf-25bbf217daae None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:18.345365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:18.346203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:18.346485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:18.346795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] PATH_INFO: `/v3/projects/8f80c3328f534e70b9079cf1f1a6b2b3` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:18.347990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f80c3328f534e70b9079cf1f1a6b2b3)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:18.349900 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:18.479460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f80c3328f534e70b9079cf1f1a6b2b3', 'request_id': 'req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.485865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f80c3328f534e70b9079cf1f1a6b2b3', 'request_id': 'req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.591698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f80c3328f534e70b9079cf1f1a6b2b3', 'request_id': 'req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.604175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] The token cache is being invalidate because project 8f80c3328f534e70b9079cf1f1a6b2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:18.604640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.605287 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.606920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1327/2638] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:18 2022] DELETE /identity/v3/projects/8f80c3328f534e70b9079cf1f1a6b2b3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:18.610903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:18.621402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a0c4e-e2ef-47f6-b755-405ca12eb031 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:18.654289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:18.679788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:18.680681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:18.680983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:18.681265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] PATH_INFO: `/v3/users/e18bbde04b7342ac895cfa8520a2d3c6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:18.683104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18bbde04b7342ac895cfa8520a2d3c6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:18.685066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:18.903760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18bbde04b7342ac895cfa8520a2d3c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.915853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18bbde04b7342ac895cfa8520a2d3c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.986647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18bbde04b7342ac895cfa8520a2d3c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.993847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18bbde04b7342ac895cfa8520a2d3c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:18.994472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18bbde04b7342ac895cfa8520a2d3c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.001955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1312/2639] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:18 2022] DELETE /identity/v3/users/e18bbde04b7342ac895cfa8520a2d3c6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:19.395367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:19.396824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:19.412325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0970cdf5-c78d-44f1-9949-d013dfaf7e1c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:19.473100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:19.478464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.478464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.478464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] PATH_INFO: `/v3/projects/f6d15fca26f34671a1a043978258bece` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.478464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d15fca26f34671a1a043978258bece)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:19.478464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:19.609768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d15fca26f34671a1a043978258bece', 'request_id': 'req-f239c625-5d5c-46d6-ac27-7b25b3027110'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.615921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d15fca26f34671a1a043978258bece', 'request_id': 'req-f239c625-5d5c-46d6-ac27-7b25b3027110'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.695221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d15fca26f34671a1a043978258bece', 'request_id': 'req-f239c625-5d5c-46d6-ac27-7b25b3027110'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.707828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:19.708901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] The token cache is being invalidate because project f6d15fca26f34671a1a043978258bece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:19.709195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.709286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:19.709842 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-f239c625-5d5c-46d6-ac27-7b25b3027110 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:19.714621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1328/2640] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:19 2022] DELETE /identity/v3/projects/f6d15fca26f34671a1a043978258bece => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:19.716120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332027-ebc6-4223-a382-a1a4c27b0c7b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.716410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332027-ebc6-4223-a382-a1a4c27b0c7b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.716766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66332027-ebc6-4223-a382-a1a4c27b0c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.724623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:19.726849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7a422-9194-473b-b57e-9546f7167dd1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:19.738425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-66332027-ebc6-4223-a382-a1a4c27b0c7b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:19.740052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-66332027-ebc6-4223-a382-a1a4c27b0c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:19.789194 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:19.790063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.790560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.790825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] PATH_INFO: `/v3/users/3cc3965dde01483495aab781b64b9860` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.805709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc3965dde01483495aab781b64b9860)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:19.807750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:19.814351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1329/2641] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:19.822045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.822335 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.822655 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.841468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:19.842745 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:19.850473 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1330/2642] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:19.855067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:19.865071 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a3d8a-ed57-4674-9bbd-2fa777036892 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:19.871380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:19.871380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.871380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.871715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.872429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:19.872736 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:19.874314 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:19.885494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1331/2643] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:24:19.890000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:19.899838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd0d61-536c-4545-958f-94ec8aa7fc19 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:19.904475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:19.905121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:19.905399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:19.905699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:19.906367 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:19.908272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.028933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1332/2644] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:19 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:20.033103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.068722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:20.070817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc3965dde01483495aab781b64b9860'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:20.083505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc3965dde01483495aab781b64b9860'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:20.098691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c5b3c-2ae9-401b-bddc-3f515bf55f29 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.103457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.104121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.104400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.105323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.106060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.108027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.147208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc3965dde01483495aab781b64b9860'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:20.155271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc3965dde01483495aab781b64b9860'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:20.155927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-372d2957-8281-4858-ac80-feb3e71d3a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc3965dde01483495aab781b64b9860'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:20.162722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1313/2645] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:19 2022] DELETE /identity/v3/users/3cc3965dde01483495aab781b64b9860 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:20.168056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.168416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.168703 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.186012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:20.187248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.207658 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1333/2646] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:20 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:20.208711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.210336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.222847 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee3f57-59bd-4f86-bf09-f3b5f2dd4048 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.248603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1314/2647] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:20.258544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.260671 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.276000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:20.296610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.296610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.296610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.296610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.297063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.297063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.299617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1334/2648] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:20.303448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.313634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77068b99-bb22-4aea-9a15-783ebc06cf77 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.318974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.319619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.319779 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48e782-297f-423a-81bf-b8063e93af6d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.319918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.320191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.320899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.322478 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.323571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.324214 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.324496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.324772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.332572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1335/2649] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:24:20.333892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.335411 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856edeec-e846-41b5-a370-3554163f1e2e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.336657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.345892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1315/2650] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:20.346705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1935d0e-e80e-4686-afa2-c8613d9a3713 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.351365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.352244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.352823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.353571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] PATH_INFO: `/v3/projects/016e0499b87e4bdf8b4ca1e019c166bc/users/9e8b94d7540f4f949e602e02a09f6a98/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.372611 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=016e0499b87e4bdf8b4ca1e019c166bc, user_id=9e8b94d7540f4f949e602e02a09f6a98, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.375304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.490597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1336/2651] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:20 2022] PUT /identity/v3/projects/016e0499b87e4bdf8b4ca1e019c166bc/users/9e8b94d7540f4f949e602e02a09f6a98/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:20.501617 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.501906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.502185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.528592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] MFA Rules not processed for user `9e8b94d7540f4f949e602e02a09f6a98`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:20.529760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.592413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:20.603270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1316/2652] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:20.616365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.618202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.678337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806833-eb6f-4f59-bedd-b4acdf989c4e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.682142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.683488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.683775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.684055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.693275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.694795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:20.704517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1337/2653] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:20.783382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:20.784982 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:20.876291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf99cf5-9b29-41c7-8dd9-1d9bb4fdc497 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:20.880853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:20.881733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:20.882013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:20.882287 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] PATH_INFO: `/v3/projects/e333ec21a1cf434285f12f29189344ea` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:20.893574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e333ec21a1cf434285f12f29189344ea)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:20.895549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:21.029690 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e333ec21a1cf434285f12f29189344ea', 'request_id': 'req-a6abd779-2398-4c8c-ac7b-bcd5056be092'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.135865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e333ec21a1cf434285f12f29189344ea', 'request_id': 'req-a6abd779-2398-4c8c-ac7b-bcd5056be092'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.144447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e333ec21a1cf434285f12f29189344ea', 'request_id': 'req-a6abd779-2398-4c8c-ac7b-bcd5056be092'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.155449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] The token cache is being invalidate because project e333ec21a1cf434285f12f29189344ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:21.155739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.156399 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.157971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1317/2654] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:20 2022] DELETE /identity/v3/projects/e333ec21a1cf434285f12f29189344ea => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:21.162064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:21.174103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a11bd-c7e8-4bc2-af16-7b2f25d456d5 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:21.232969 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:21.233874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:21.234155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:21.234458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] PATH_INFO: `/v3/users/a4d7c7dd03324c0e904a77b0cce6137c` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:21.249321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d7c7dd03324c0e904a77b0cce6137c)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:21.252057 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:21.578225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d7c7dd03324c0e904a77b0cce6137c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.579406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:21.591486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d7c7dd03324c0e904a77b0cce6137c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.667887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d7c7dd03324c0e904a77b0cce6137c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.675435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d7c7dd03324c0e904a77b0cce6137c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.679897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a915f472-56e2-4c84-a0b4-b0c9ceb2afc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d7c7dd03324c0e904a77b0cce6137c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:21.682710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1338/2655] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:21 2022] DELETE /identity/v3/users/a4d7c7dd03324c0e904a77b0cce6137c => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:22.143149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:22.144623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:22.158033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd779-2398-4c8c-ac7b-bcd5056be092 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:22.215758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:22.216551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.216822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.217091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] PATH_INFO: `/v3/projects/87b5c153431b4f0eba60340c81a2e2ef` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.227567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b5c153431b4f0eba60340c81a2e2ef)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:22.229494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:22.241249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:22.341682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b5c153431b4f0eba60340c81a2e2ef', 'request_id': 'req-115fa9b6-ac35-4e12-b373-92c88cd59518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:22.427255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b5c153431b4f0eba60340c81a2e2ef', 'request_id': 'req-115fa9b6-ac35-4e12-b373-92c88cd59518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:22.434976 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b5c153431b4f0eba60340c81a2e2ef', 'request_id': 'req-115fa9b6-ac35-4e12-b373-92c88cd59518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:22.445238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] The token cache is being invalidate because project 87b5c153431b4f0eba60340c81a2e2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:22.445525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:22.446159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-115fa9b6-ac35-4e12-b373-92c88cd59518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:22.447730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1318/2656] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:22 2022] DELETE /identity/v3/projects/87b5c153431b4f0eba60340c81a2e2ef => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:22.452932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.453229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.453511 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.474840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:22.476011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:22.540437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1339/2657] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:22.549847 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.550133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.550422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.570078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:22.571581 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:22.581199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1319/2658] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:22.585628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:22.598272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ca2be-32c1-4db0-a9a1-4b70ad9903db None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:22.603350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:22.604016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.604303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.604587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.606434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:22.606858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:22.608533 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:22.621823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1340/2659] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 15:24:22.626109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:22.635599 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c9d1-98e1-412d-8a6c-851e788895f4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:22.640788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:22.641446 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.641748 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.642005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.642741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:22.644652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:22.784296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1320/2660] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:22 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:22.788651 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:22.799889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a85906-3c4b-44c9-adb2-fdb69e6961be admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:22.864467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:22.865140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:22.865427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:22.865730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:22.866447 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:22.868441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:22.984197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1341/2661] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:22 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:23.036343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.048522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99ad1a-a88d-4560-b1ff-e8a1be65897e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.053704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.054364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.054684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.054959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.056310 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.059154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.071208 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1321/2662] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:23.075948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.086597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14509-c532-4210-bcf9-b22fb4e11dd0 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.094813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.095581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.095870 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.096150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.096958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.102960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ca2bc-0b87-4e07-bc6b-3e5d676e5b14 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.111027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1342/2663] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:24:23.115329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.125701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61eb33-9da3-434d-ae94-2e682a60529f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.130356 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.148805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.148805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.148805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] PATH_INFO: `/v3/projects/600a7686ff0b4d10b4a99e4a126153ca/users/41e9f70da56d47068e1c1ba2580f2111/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.148805 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] RBAC: Authorizing `identity:create_grant(project_id=600a7686ff0b4d10b4a99e4a126153ca, user_id=41e9f70da56d47068e1c1ba2580f2111, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.150186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.251963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1322/2664] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:23 2022] PUT /identity/v3/projects/600a7686ff0b4d10b4a99e4a126153ca/users/41e9f70da56d47068e1c1ba2580f2111/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:23.262934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.263226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.263502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.289909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] MFA Rules not processed for user `41e9f70da56d47068e1c1ba2580f2111`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:23.291070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:23.309465 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:23.355011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1343/2665] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:23.359587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.422200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31938780-306a-4727-9a21-029f4fde78de admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.426801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.427457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.427741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.428019 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.428708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.430628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.521065 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1323/2666] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:23 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:23.525206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.536738 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1951583c-e47f-42d2-a7f4-e5013d440bc0 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.594089 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.594771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.594976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.595251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.595930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.597909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.742484 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1344/2667] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:23 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:23.747651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.758365 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29120158-9218-471c-aef6-8b29f74c9fdb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.763111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.763772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.764311 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.764589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.765313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.766943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.777899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1324/2668] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:23.782177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.792190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868f5b-3155-456b-b2a4-fe8dd1408b84 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.797733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.798636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.798944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.815504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] PATH_INFO: `/v3/projects/b879849f5aa848d084abb6312993b318/users/b969084cdadb48fa94fd23f7c18e706c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.815504 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b879849f5aa848d084abb6312993b318, user_id=b969084cdadb48fa94fd23f7c18e706c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.817620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.896659 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1345/2669] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:23 2022] PUT /identity/v3/projects/b879849f5aa848d084abb6312993b318/users/b969084cdadb48fa94fd23f7c18e706c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:23.900519 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.911171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006041-6275-49bf-9b21-d76a795db6d8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:23.915660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:23.969545 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:23.970218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:23.970495 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:23.970806 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:23.971532 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:23.973188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbca1ba-792c-4f43-8b33-40435394d757 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:23.987067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1325/2670] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:23.988268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:23.999063 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6e7f6-f890-48d4-b7e0-647166becde5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:24.003802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:24.004715 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:24.004995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:24.005272 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] PATH_INFO: `/v3/projects/b879849f5aa848d084abb6312993b318/users/b969084cdadb48fa94fd23f7c18e706c/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:24.007234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b879849f5aa848d084abb6312993b318, user_id=b969084cdadb48fa94fd23f7c18e706c, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:24.009914 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:24.021461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1346/2671] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:23 2022] PUT /identity/v3/projects/b879849f5aa848d084abb6312993b318/users/b969084cdadb48fa94fd23f7c18e706c/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:24.032373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:24.032663 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:24.032938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:24.059624 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] MFA Rules not processed for user `b969084cdadb48fa94fd23f7c18e706c`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:24.060865 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:24.135227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1326/2672] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:24.154018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:24.154018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:24.220683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154bed05-8056-44b7-a2e1-bc6673c1cfea admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:24.224519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:24.225173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:24.225460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:24.225759 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:24.236467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:24.237961 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:24.248255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1347/2673] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:25.172321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:25.181422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d2842-0c87-4092-997f-78fc7646a83b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:25.186388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b969084cdadb48fa94fd23f7c18e706c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b879849f5aa848d084abb6312993b318', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:25.187076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:25.187359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:25.187642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:25.188370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:25.188684 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:25.190362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:25.209278 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1327/2674] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:25.210427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:25.220370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9724f2-ebde-48e4-9799-4d4eab4fe7f2 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:25.224873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b969084cdadb48fa94fd23f7c18e706c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b879849f5aa848d084abb6312993b318', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:25.225516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:25.225798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:25.226076 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:25.226792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:25.228665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:25.242245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:25.319902 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1348/2675] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:25 2022] POST /identity/v3/projects => generated 354 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:25.324558 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:25.335157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e1fb-0079-447a-95d6-38cab97bacf7 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:25.394890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b969084cdadb48fa94fd23f7c18e706c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b879849f5aa848d084abb6312993b318', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:25.395578 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:25.395869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:25.396152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:25.396879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:25.397193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:25.398824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:25.410701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1328/2676] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:25.446242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:25.457190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc39802-ebd5-4d2a-b83f-88aad11d54a5 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:25.461767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b969084cdadb48fa94fd23f7c18e706c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b879849f5aa848d084abb6312993b318', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:25.462645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:25.462943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:25.463223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] PATH_INFO: `/v3/projects/c83867f43fa848fea10d011b68ab3cf0` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:25.464393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c83867f43fa848fea10d011b68ab3cf0)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:25.466604 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:25.641331 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83867f43fa848fea10d011b68ab3cf0', 'request_id': 'req-42a8dacd-3362-442d-ad8b-b095300fa38c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:25.648118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83867f43fa848fea10d011b68ab3cf0', 'request_id': 'req-42a8dacd-3362-442d-ad8b-b095300fa38c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:25.748134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83867f43fa848fea10d011b68ab3cf0', 'request_id': 'req-42a8dacd-3362-442d-ad8b-b095300fa38c'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:25.759223 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] The token cache is being invalidate because project c83867f43fa848fea10d011b68ab3cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:25.759507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:25.760143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-42a8dacd-3362-442d-ad8b-b095300fa38c tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:25.761893 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1349/2677] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:25 2022] DELETE /identity/v3/projects/c83867f43fa848fea10d011b68ab3cf0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:26.414128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:26.415974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:26.429908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:26.481330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89636c17-c0a5-45fc-9ae9-d0a53efc6b17 tempest-VolumeQuotaClassesTest-167348216 tempest-VolumeQuotaClassesTest-167348216-project-admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:26.537203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:26.538074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:26.538359 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:26.538667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] PATH_INFO: `/v3/users/41e9f70da56d47068e1c1ba2580f2111` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:26.539844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e9f70da56d47068e1c1ba2580f2111)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:26.541762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:26.812727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e9f70da56d47068e1c1ba2580f2111'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:26.824284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e9f70da56d47068e1c1ba2580f2111'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:26.907303 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e9f70da56d47068e1c1ba2580f2111'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:26.914314 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e9f70da56d47068e1c1ba2580f2111'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:26.914969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e9f70da56d47068e1c1ba2580f2111'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:26.921626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1329/2678] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:26 2022] DELETE /identity/v3/users/41e9f70da56d47068e1c1ba2580f2111 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:26.926072 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848519-628b-4930-84b4-594adde9af90 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:26.926563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848519-628b-4930-84b4-594adde9af90 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:26.926711 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5848519-628b-4930-84b4-594adde9af90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:26.944239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a5848519-628b-4930-84b4-594adde9af90 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:26.945453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5848519-628b-4930-84b4-594adde9af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:27.003638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1350/2679] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:27.014012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:27.015694 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:27.056454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:27.071876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80242231-3ff3-4bd9-b18a-1e1f94894ca5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:27.076072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:27.076712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:27.076993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:27.077268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:27.086757 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:27.088200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:27.098812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1330/2680] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:27.509485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848519-628b-4930-84b4-594adde9af90 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:27.510956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a5848519-628b-4930-84b4-594adde9af90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:27.523635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5848519-628b-4930-84b4-594adde9af90 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:27.529550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:27.529550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:27.529777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:27.529945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] PATH_INFO: `/v3/projects/600a7686ff0b4d10b4a99e4a126153ca` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:27.531168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600a7686ff0b4d10b4a99e4a126153ca)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:27.533090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:27.673913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600a7686ff0b4d10b4a99e4a126153ca', 'request_id': 'req-c7783fd4-8fc0-4108-a455-77dfc85f7b63'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:27.680054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600a7686ff0b4d10b4a99e4a126153ca', 'request_id': 'req-c7783fd4-8fc0-4108-a455-77dfc85f7b63'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:27.765364 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600a7686ff0b4d10b4a99e4a126153ca', 'request_id': 'req-c7783fd4-8fc0-4108-a455-77dfc85f7b63'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:27.772780 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:27.777338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] The token cache is being invalidate because project 600a7686ff0b4d10b4a99e4a126153ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:27.777338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:27.778009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:27.779694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1351/2681] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:27 2022] DELETE /identity/v3/projects/600a7686ff0b4d10b4a99e4a126153ca => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:27.783814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:27.794255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad347aa-8d37-44a2-b246-72ab35933abc service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:27.850423 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:27.851454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:27.851738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:27.852012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] PATH_INFO: `/v3/users/b969084cdadb48fa94fd23f7c18e706c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:27.853213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b969084cdadb48fa94fd23f7c18e706c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:27.855308 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:28.013725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b969084cdadb48fa94fd23f7c18e706c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.025780 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b969084cdadb48fa94fd23f7c18e706c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.096265 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b969084cdadb48fa94fd23f7c18e706c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.103597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b969084cdadb48fa94fd23f7c18e706c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.104237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f9fb9e7e-7d5c-425d-ab92-6ce1cc89dd3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b969084cdadb48fa94fd23f7c18e706c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.110347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1331/2682] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:27 2022] DELETE /identity/v3/users/b969084cdadb48fa94fd23f7c18e706c => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:28.583540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:28.585105 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:28.598616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7783fd4-8fc0-4108-a455-77dfc85f7b63 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:28.657580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:28.658453 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:28.658765 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:28.659046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] PATH_INFO: `/v3/projects/b879849f5aa848d084abb6312993b318` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:28.660221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b879849f5aa848d084abb6312993b318)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:28.662140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:28.766276 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b879849f5aa848d084abb6312993b318', 'request_id': 'req-43a40c4d-8203-4600-b0a6-0a861c58fb1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.772498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b879849f5aa848d084abb6312993b318', 'request_id': 'req-43a40c4d-8203-4600-b0a6-0a861c58fb1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.851175 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b879849f5aa848d084abb6312993b318', 'request_id': 'req-43a40c4d-8203-4600-b0a6-0a861c58fb1f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.862502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] The token cache is being invalidate because project b879849f5aa848d084abb6312993b318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:28.862815 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.863433 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-43a40c4d-8203-4600-b0a6-0a861c58fb1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:28.865066 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1352/2683] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:28 2022] DELETE /identity/v3/projects/b879849f5aa848d084abb6312993b318 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:28.870053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861d1-0732-485c-930e-975441a2bbbf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:28.870493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861d1-0732-485c-930e-975441a2bbbf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:28.870603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874861d1-0732-485c-930e-975441a2bbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:28.889245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-874861d1-0732-485c-930e-975441a2bbbf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:28.890752 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-874861d1-0732-485c-930e-975441a2bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:28.942175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:28.951371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1332/2684] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:28.960308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:28.960601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:28.960878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:28.979576 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:28.980788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:28.988977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1353/2685] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:28.993277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874861d1-0732-485c-930e-975441a2bbbf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.003499 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874861d1-0732-485c-930e-975441a2bbbf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.008089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.008744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.009028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.009384 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.010032 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:29.010347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.011988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.024416 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1333/2686] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:29.029567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.038874 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195fc0f-bcc1-45bd-8aea-9c37910a61ac None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.043546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.044370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.044661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.044941 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.045640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.047593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.161181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1354/2687] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:29 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:29.165186 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.175309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506ad7-b477-4860-9a7f-5fd368d2877a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.238134 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.240236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.240236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.240236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.240236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.242362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.361313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1334/2688] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:29 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:29.365563 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.379010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265be236-9db3-4fb8-bb7d-c2a9f9252f06 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.384107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.384868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.385203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.385550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.386356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.389001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.400483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1355/2689] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:29.406741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.417112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209292f6-e893-44ce-b61e-11b4e07e8767 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.422023 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.422723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.423005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.423283 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.424040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.425642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af8052-b909-4a25-a637-0076fc0b8b79 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.436183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1335/2690] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:29.440303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.451928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48221c4-acdb-497a-a6be-49cc0bdab154 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.457027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.457909 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.458196 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.458476 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] PATH_INFO: `/v3/projects/1f4a9f84a03c4c869bdd57eb1978a027/users/6457b4237c0f4a3e9de932791640e876/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.474036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4a9f84a03c4c869bdd57eb1978a027, user_id=6457b4237c0f4a3e9de932791640e876, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.476719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.560976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1356/2691] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:29 2022] PUT /identity/v3/projects/1f4a9f84a03c4c869bdd57eb1978a027/users/6457b4237c0f4a3e9de932791640e876/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:29.571603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.571898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.572199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.598497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] MFA Rules not processed for user `6457b4237c0f4a3e9de932791640e876`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:29.599739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:29.667835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1336/2692] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:29.679457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.681168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:29.712512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:29.740009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0f8d7-18d8-4693-884a-44b1603a09e6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.743667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.744308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.744588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.744860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.758862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.760376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a466c-4ab0-4c5b-ac32-a191d2529563 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:29.771697 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1357/2693] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:29.792106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:29.849637 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:29.855057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c2fd0-09ad-4905-ae8c-d807c8c1e36c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:29.860812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:29.861722 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:29.862000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:29.862269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] PATH_INFO: `/v3/users/6457b4237c0f4a3e9de932791640e876` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:29.863496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6457b4237c0f4a3e9de932791640e876)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:29.865410 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:30.128720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6457b4237c0f4a3e9de932791640e876'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.140822 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6457b4237c0f4a3e9de932791640e876'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.222601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6457b4237c0f4a3e9de932791640e876'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.229950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6457b4237c0f4a3e9de932791640e876'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.230600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6457b4237c0f4a3e9de932791640e876'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.236981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1337/2694] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:29 2022] DELETE /identity/v3/users/6457b4237c0f4a3e9de932791640e876 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:30.241421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:30.241710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:30.241987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:30.259757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:30.260953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:30.318676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1358/2695] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:30.328997 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:30.330430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:30.390897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8d9ef-3062-4f50-a792-fc4c1e632cdc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:30.394542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:30.395199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:30.395483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:30.395761 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:30.405270 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:30.406749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f99b2a-d246-4487-9eb4-a4054ec882d5 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:30.416189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1338/2696] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:30.858921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:30.860406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:30.872869 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c9fbd-a43c-4513-af0d-966e4ab160fe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:30.885636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:30.885636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:30.885636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:30.885636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] PATH_INFO: `/v3/projects/1f4a9f84a03c4c869bdd57eb1978a027` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:30.886435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4a9f84a03c4c869bdd57eb1978a027)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:30.886435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:30.988764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4a9f84a03c4c869bdd57eb1978a027', 'request_id': 'req-64468296-22e7-47a3-839a-e58002ec84d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:30.994913 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4a9f84a03c4c869bdd57eb1978a027', 'request_id': 'req-64468296-22e7-47a3-839a-e58002ec84d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:31.062757 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4a9f84a03c4c869bdd57eb1978a027', 'request_id': 'req-64468296-22e7-47a3-839a-e58002ec84d0'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:31.073953 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] The token cache is being invalidate because project 1f4a9f84a03c4c869bdd57eb1978a027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:31.074246 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:31.074879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-64468296-22e7-47a3-839a-e58002ec84d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:31.076428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1359/2697] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:30 2022] DELETE /identity/v3/projects/1f4a9f84a03c4c869bdd57eb1978a027 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:31.081852 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.082245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.082561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.100695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:31.101906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:31.161886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1339/2698] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:31.170011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.170299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.170623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.189172 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:31.190407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:31.198312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1360/2699] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:31.202474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.211986 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9743a-18a5-4e79-8672-121aeb2f3f7f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.216569 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.217203 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.217480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.217751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.218459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:31.218797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.220382 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.232109 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1340/2700] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:31.236432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.245740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0351d-8ef0-4434-bcfb-26743ecfe7cf None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.251400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.252046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.252329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.252606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.253290 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.255231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.347275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1361/2701] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:31 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:31.351590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.359067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:31.363078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028dadb-331e-4063-b652-124bda5f1b16 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.420950 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.421560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.422060 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.422385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.423316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.425640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.537192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1341/2702] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:31 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:31.541143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.552868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc87087-23f7-457c-9a2f-b5f022e25a97 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.557664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.558446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.558771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.559053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.560018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.561684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.572191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1362/2703] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:31.576114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.586215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee932bd-b8c3-4347-b094-33991584c370 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.591450 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.592107 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.592388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.592666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.593388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.595007 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be9ea6-24ac-4318-b120-81576d893570 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.604827 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1342/2704] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:31.609580 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.619380 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55913bb7-bfe1-4405-8d78-d9bea067a604 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.624038 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.624895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.625174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.625443 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] PATH_INFO: `/v3/projects/7f57922f551847228078aee6767bb3e9/users/9280b5654602460387a59a11e7536eb1/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.636182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:31.640425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f57922f551847228078aee6767bb3e9, user_id=9280b5654602460387a59a11e7536eb1, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.643179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.760315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1363/2705] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:31 2022] PUT /identity/v3/projects/7f57922f551847228078aee6767bb3e9/users/9280b5654602460387a59a11e7536eb1/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:31.772256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.772546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.772824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.800551 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] MFA Rules not processed for user `9280b5654602460387a59a11e7536eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:31.801759 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:31.867100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1343/2706] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:31.877600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:31.879461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:31.937191 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53682-7cff-4a35-8abe-85ce3da616c9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:31.940856 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:31.941502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:31.941781 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:31.942054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:31.950417 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:31.952483 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5837446-e258-4e1c-bb0b-93f3fc3ae5a3 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:31.961623 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1364/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:37.123932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:37.188974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc40d48-2699-4658-aa9c-8ca712435ad7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:37.193830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:37.194754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.195043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.195322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] PATH_INFO: `/v3/users/9280b5654602460387a59a11e7536eb1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.196521 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9280b5654602460387a59a11e7536eb1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:37.198536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:37.212394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:37.350947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9280b5654602460387a59a11e7536eb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.372309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9280b5654602460387a59a11e7536eb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.417453 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9280b5654602460387a59a11e7536eb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.425263 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9280b5654602460387a59a11e7536eb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.425935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9280b5654602460387a59a11e7536eb1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:37.432131 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1344/2708] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:37 2022] DELETE /identity/v3/users/9280b5654602460387a59a11e7536eb1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:37.436591 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.437060 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.437143 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.454778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:37.456593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:37.518761 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1365/2709] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:37.528648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:37.530392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:37.588001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba91fb-0300-43af-9b81-2467a7978a51 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:37.591634 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:37.592284 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:37.592567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:37.592845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:37.602175 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:37.603656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e44e58-b098-47fe-a055-d5e08df5a6df service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:37.613640 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1345/2710] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:38.013806 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.015261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:38.028216 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4b5ab-741d-4727-8193-c78e65df1c96 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.032846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.033933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.034213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.034485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] PATH_INFO: `/v3/projects/7f57922f551847228078aee6767bb3e9` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.035988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f57922f551847228078aee6767bb3e9)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.037947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.189680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f57922f551847228078aee6767bb3e9', 'request_id': 'req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.196425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f57922f551847228078aee6767bb3e9', 'request_id': 'req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.280892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f57922f551847228078aee6767bb3e9', 'request_id': 'req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.292876 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] The token cache is being invalidate because project 7f57922f551847228078aee6767bb3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:38.293303 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.293928 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-a7bf79d1-78a4-4ca0-b3d8-4d4f707d96bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:38.295503 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1366/2711] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:38 2022] DELETE /identity/v3/projects/7f57922f551847228078aee6767bb3e9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:38.302620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.303112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.303112 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.322869 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:38.325313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:38.384903 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1346/2712] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:38.392858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.393142 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.393416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.411320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:38.412514 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:38.421515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1367/2713] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:38.424818 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.435286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39ac4b-bdc9-4cb1-8948-6f6a031fa555 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.439870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.440524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.440807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.441084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.442507 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:38.442849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.444448 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.456474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1347/2714] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:38.460930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.470052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d5a4-65f4-43bc-9d44-32de4b55393f None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.474925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.475564 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.475844 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.476118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.476805 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.478735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.482714 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:38.624981 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1368/2715] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:38 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:38.629914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.639834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cfa4-3d2e-45b0-be5d-b8258ce47b6d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.658398 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:38.695857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.696523 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.696808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.697087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.697782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.699735 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.812167 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1348/2716] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:38 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:38.816205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.826668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a9c4a-cf95-4d33-ac0d-c7a9438d1494 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.831491 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.832152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.832432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.832708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.833497 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.835119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.845382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1369/2717] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:38.849835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.860850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddb315-a086-4f4f-bf4d-d918a90af52b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.866451 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.867156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.867468 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.867723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.868460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.870056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74641dde-5102-4014-9044-53d64f196b19 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:38.881095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1349/2718] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:38.885173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:38.895648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a031a-6b77-44f1-93c6-990dc8bc7e0d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:38.900304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:38.901174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:38.901458 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:38.901734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] PATH_INFO: `/v3/projects/6a1633d3a76f4a54876b11fdba57636c/users/f7793fa995e64744b2e5ee5cf1a91514/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:38.917467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1633d3a76f4a54876b11fdba57636c, user_id=f7793fa995e64744b2e5ee5cf1a91514, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:38.920158 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.018778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1370/2719] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:38 2022] PUT /identity/v3/projects/6a1633d3a76f4a54876b11fdba57636c/users/f7793fa995e64744b2e5ee5cf1a91514/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:39.033318 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fd95-5371-4ac2-97c1-c72a779081cb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.033607 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fd95-5371-4ac2-97c1-c72a779081cb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.033888 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258fd95-5371-4ac2-97c1-c72a779081cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.052629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4258fd95-5371-4ac2-97c1-c72a779081cb None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:39.053832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4258fd95-5371-4ac2-97c1-c72a779081cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:39.113510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1350/2720] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:39.123107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:39.124873 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:39.182777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a207f1b-d01c-4373-8f90-4389d6de252e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:39.187109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:39.187744 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:39.188022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:39.188295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:39.197404 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:39.198862 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:39.208601 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1371/2721] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:45.200047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.200625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.200625 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.228856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] MFA Rules not processed for user `f7793fa995e64744b2e5ee5cf1a91514`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:45.230113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:45.303926 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1351/2722] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:45.308327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.319268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21a88-2e1a-4410-831f-56827512f561 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.325510 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.326385 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.326705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.326990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.327699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.329657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.478315 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1372/2723] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:45 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:45.482574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.494899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ccc97-6602-47b8-b97a-4af6dd7ab454 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.526815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:45.556053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.556766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.557049 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.557330 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.558020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.560251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.666087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1352/2724] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:45 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:45.670002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.681151 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a99d9-0fba-40f9-8600-fdf6b1f953fa admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.685751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.690619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.690619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.690619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.690619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.690619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.705490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1373/2725] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:45.706368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.716979 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856e025-eecd-4977-bd38-7bd937b85d38 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.721595 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.722478 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.722796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.723055 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] PATH_INFO: `/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331/users/deb743191e1b42b68157737319bd1c5f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.738667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2c0d736c5f4236b9b1ef7077a1f331, user_id=deb743191e1b42b68157737319bd1c5f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.741912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.824653 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1353/2726] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:45 2022] PUT /identity/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331/users/deb743191e1b42b68157737319bd1c5f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:45.828619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.839312 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e176b-aa9b-4621-871a-17d14812ca90 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.898356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.899046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.899328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.899606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.900318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.901934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.915434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1374/2727] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:45.916776 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:45.928299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585f249-6287-49da-951f-56c47b31d619 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:45.932968 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:45.935174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:45.935174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:45.935174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] PATH_INFO: `/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331/users/deb743191e1b42b68157737319bd1c5f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:45.936854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2c0d736c5f4236b9b1ef7077a1f331, user_id=deb743191e1b42b68157737319bd1c5f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:45.939530 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b345d2-0015-4902-b438-32148f9c5f6f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:45.950240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1354/2728] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:45 2022] PUT /identity/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331/users/deb743191e1b42b68157737319bd1c5f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:50.452678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:50.455231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:50.469578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687863ea-5f4f-48d6-8c54-8561222a436d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:50.494376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:50.555182 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:50.556090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:50.556442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:50.556791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] PATH_INFO: `/v3/users/9e8b94d7540f4f949e602e02a09f6a98` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:50.558092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e8b94d7540f4f949e602e02a09f6a98)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:50.560232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:50.751676 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e8b94d7540f4f949e602e02a09f6a98'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:50.766396 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e8b94d7540f4f949e602e02a09f6a98'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:50.922215 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e8b94d7540f4f949e602e02a09f6a98'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:50.932866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e8b94d7540f4f949e602e02a09f6a98'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:50.933707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e8b94d7540f4f949e602e02a09f6a98'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:50.940624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1375/2729] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:24:50 2022] DELETE /identity/v3/users/9e8b94d7540f4f949e602e02a09f6a98 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:50.946072 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:50.946573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:50.946652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:50.967944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:50.969948 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:51.057588 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1355/2730] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:51.068537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:51.069795 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:51.180264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebfc52-f7c8-4c16-95f7-977ed317fb08 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:51.184118 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:51.184792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:51.185078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:51.185383 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:51.196176 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:51.197647 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf68418-7537-4330-ba15-a6d5362ed9e1 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:51.208673 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1376/2731] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:51.910826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:51.913824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:51.955760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c736a2-f415-420b-ba37-2316b899bcf2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:51.967542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:51.968490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:51.968825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:51.970069 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] PATH_INFO: `/v3/projects/016e0499b87e4bdf8b4ca1e019c166bc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:51.975135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016e0499b87e4bdf8b4ca1e019c166bc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:51.975135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.080778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.081294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.081370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.120434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] MFA Rules not processed for user `deb743191e1b42b68157737319bd1c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:52.121948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:52.127473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016e0499b87e4bdf8b4ca1e019c166bc', 'request_id': 'req-a78d1349-7f36-45df-83ff-bc971fcb4567'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:52.184791 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:52.218983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016e0499b87e4bdf8b4ca1e019c166bc', 'request_id': 'req-a78d1349-7f36-45df-83ff-bc971fcb4567'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:52.230003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016e0499b87e4bdf8b4ca1e019c166bc', 'request_id': 'req-a78d1349-7f36-45df-83ff-bc971fcb4567'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:52.233224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1377/2732] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:52.251728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.251900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] The token cache is being invalidate because project 016e0499b87e4bdf8b4ca1e019c166bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:24:52.253130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:52.254549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:52.257656 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a78d1349-7f36-45df-83ff-bc971fcb4567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:24:52.259418 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1356/2733] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:24:51 2022] DELETE /identity/v3/projects/016e0499b87e4bdf8b4ca1e019c166bc => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:24:52.273642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0956-4db9-4fee-b2ff-cd288516ad9d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.273936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0956-4db9-4fee-b2ff-cd288516ad9d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.274215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a0956-4db9-4fee-b2ff-cd288516ad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.290639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:52.299022 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-898a0956-4db9-4fee-b2ff-cd288516ad9d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:52.300268 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-898a0956-4db9-4fee-b2ff-cd288516ad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:52.361996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7589-8ddb-4072-aabd-bbf178474349 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.372490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.373166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.373441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.373716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.377306 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:52.403029 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1357/2734] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:24:52.405463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.405463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.405463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.431083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:52.432307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:52.443126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1358/2735] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:52.447297 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.462959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bea16f-30d8-44af-9c8e-b4d080c6b83e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.465221 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.467428 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.469564 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.470376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.471312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.471712 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.472555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:24:52.473162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.475859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.484643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1378/2736] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:24:52.491244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1359/2737] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:24:52.495588 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.511309 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6344ba6-f5ac-4612-a06c-e5e2aebcf894 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.518043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.518043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.518043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.518043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.518374 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.520304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.627722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1379/2738] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:24:52 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:24:52.629335 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.662050 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab30d2c-44fc-4b3d-810e-8307fcd57025 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.725732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.726464 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.727130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.727130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.727794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.730272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.838135 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1360/2739] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:24:52 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:24:52.842179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.853178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264c0df-343d-4430-9730-45d639a59fbf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.859653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.860375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.862362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.862362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.862362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.863359 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.875641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1380/2740] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:52.879550 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.890272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a2ddc-f877-4f60-b8e5-ad8a45d6849c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.895526 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.896313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.896590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.896952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.897793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.899454 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb71b46-b85a-490a-9754-91b1996e7023 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:52.913304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1361/2741] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:24:52.915240 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:52.925257 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffea14-103c-4546-ace2-961e71086dbe admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:52.930735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:52.931755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:52.932141 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:52.932461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] PATH_INFO: `/v3/projects/d3628a28b6ca4bfd8fd79f85712ff92d/users/f6f57789b3b74ce7ba76fd808c0e07d6/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:52.949201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3628a28b6ca4bfd8fd79f85712ff92d, user_id=f6f57789b3b74ce7ba76fd808c0e07d6, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:52.951897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:53.034970 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1381/2742] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:24:52 2022] PUT /identity/v3/projects/d3628a28b6ca4bfd8fd79f85712ff92d/users/f6f57789b3b74ce7ba76fd808c0e07d6/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:24:53.037395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94580f-d93b-4ac7-bb6b-ba14a4003a7f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:53.041376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94580f-d93b-4ac7-bb6b-ba14a4003a7f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:53.041376 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94580f-d93b-4ac7-bb6b-ba14a4003a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:53.070407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cb94580f-d93b-4ac7-bb6b-ba14a4003a7f None None] MFA Rules not processed for user `f6f57789b3b74ce7ba76fd808c0e07d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:53.071686 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cb94580f-d93b-4ac7-bb6b-ba14a4003a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:53.131252 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:53.140623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1362/2743] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:24:53.153949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:24:53.156489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:53.163721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:24:53.244042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1f570-f840-4dab-99f0-3eba00ac2417 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:24:53.247688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:24:53.248332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:53.248610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:53.248884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:53.257684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:24:53.259262 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:24:53.268967 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1382/2744] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:24:55.593821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:24:55.594375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:24:55.594465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:24:55.622557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] MFA Rules not processed for user `f7793fa995e64744b2e5ee5cf1a91514`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:24:55.624371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:24:55.690176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1363/2745] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:03.487617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=33 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:05.682700 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=27 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:26.189573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:26.191059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:26.276188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173906ed-51b8-48ea-928c-b0567b944659 service glance] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:26.279925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:26.280570 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:26.280865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:26.281114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:26.289317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:26.300162 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:26.301592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:26.309621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1383/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:27.179386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:27.180801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:27.252837 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd4b4b-c5d4-4eac-a319-626843eee61e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:27.258191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:27.258881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:27.259164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:27.264229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:27.264229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:27.274371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:27.275993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:27.286737 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1364/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:32.981750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:32.983348 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:32.995852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a498ba-87d3-45a7-a137-18c1d1d96e4d service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:32.999663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:33.000318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:33.000735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:33.001012 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:33.003522 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:33.013319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:33.014775 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:33.024055 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1384/2748] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:37.281644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:40.935787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:40.937316 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:40.994204 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2ea9-6a96-457a-a6b8-54f9082f5ed7 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:40.997760 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:40.998397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:40.998710 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:40.999002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:41.000762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:41.058866 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:41.060320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:41.070181 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1365/2749] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:43.020083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=32 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:50.390286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:50.392506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:50.490531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844cb7b-5469-470e-88f3-0681cd6a3032 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:50.496923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:50.497574 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:50.497855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:50.498128 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:50.499926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:50.514283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:50.515912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c0afc-a41c-4d36-a23b-1f1cd51887d5 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:50.528251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1385/2750] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:51.065094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=48 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:52.683457 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:52.760281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e521a0-3d72-4375-ac13-818ad442a472 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:52.764952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:52.765833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:52.766114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:52.766391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] PATH_INFO: `/v3/users/f6f57789b3b74ce7ba76fd808c0e07d6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:52.767670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6f57789b3b74ce7ba76fd808c0e07d6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:52.769618 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:53.050811 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6f57789b3b74ce7ba76fd808c0e07d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:53.063598 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6f57789b3b74ce7ba76fd808c0e07d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:53.185255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6f57789b3b74ce7ba76fd808c0e07d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:53.192951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6f57789b3b74ce7ba76fd808c0e07d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:53.193601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6f57789b3b74ce7ba76fd808c0e07d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:53.200322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1366/2751] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:25:52 2022] DELETE /identity/v3/users/f6f57789b3b74ce7ba76fd808c0e07d6 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:25:53.204826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:53.205358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:53.205438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:53.223793 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:25:53.224996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:53.284295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1386/2752] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:25:53.293967 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:53.295452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:53.352395 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7970f5-1c7f-4891-88a8-bab54e3237fa admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:53.356001 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:53.356647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:53.356927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:53.357199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:53.366701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:53.368717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:53.377702 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1367/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:25:53.885923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:53.887390 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:53.899545 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25576-92a5-4109-bd95-d5fccdb16d49 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:53.905444 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:53.906336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:53.906643 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:53.906915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] PATH_INFO: `/v3/projects/d3628a28b6ca4bfd8fd79f85712ff92d` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:53.908095 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3628a28b6ca4bfd8fd79f85712ff92d)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:53.910005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:54.025758 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:54.027275 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:54.033011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3628a28b6ca4bfd8fd79f85712ff92d', 'request_id': 'req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:54.039313 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f93cc3-acc8-462c-93e9-d24d5d5ac4bd service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:54.044748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3628a28b6ca4bfd8fd79f85712ff92d', 'request_id': 'req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:54.044748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:54.045118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:25:54.089677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:54.090329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:54.090771 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:54.091079 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:54.092889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:54.156447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:54.157929 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:54.168267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1368/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:25:54.168884 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3628a28b6ca4bfd8fd79f85712ff92d', 'request_id': 'req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:54.180299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] The token cache is being invalidate because project d3628a28b6ca4bfd8fd79f85712ff92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:25:54.180587 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:54.182030 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:25:54.183824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1387/2755] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:25:53 2022] DELETE /identity/v3/projects/d3628a28b6ca4bfd8fd79f85712ff92d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:25:56.457946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:25:56.459583 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:56.533951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4c9b8-3878-410d-b4ee-61dc31fd4522 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:25:56.540348 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:25:56.541271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:25:56.541510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:25:56.541787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:25:56.543698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:25:56.556696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:25:56.558279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:25:56.569286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1369/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:25:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:26:04.392250 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:26:06.564542 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=80 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:01.537693 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:01.539168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:01.656415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec1a1d-d872-4b9f-ae12-9a3301e5e0f7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:01.661754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:01.662405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:01.662724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:01.663042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:01.664852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:01.676355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:01.677932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:01.688395 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1388/2757] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:11.684190 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=53 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:14.944357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:14.946188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:14.983197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3091f-9ea6-4c96-ba4c-709193865946 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:14.986889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:15.011216 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ffa980-38d2-4aa0-b284-7bd1e51f3016 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:15.017037 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1370/2758] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:16.191177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:16.200636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df2b7d-ee01-425e-aaaf-ce185378b4fc service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:16.204271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:16.204904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:16.205189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:16.205475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:16.207251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:16.221136 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:16.222530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb071b-bd9f-4dab-a59f-e0713dfdfabe service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:16.234592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1389/2759] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:25.012304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=15 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:26.229184 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:29.662156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1fde-d380-4532-9b63-862d72d29c88 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.662739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1fde-d380-4532-9b63-862d72d29c88 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.662739 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b1fde-d380-4532-9b63-862d72d29c88 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.664170 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1371/2760] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 15:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:28:29.670984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34f97-2175-4bb6-af9d-ab5ece7492c7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.671952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34f97-2175-4bb6-af9d-ab5ece7492c7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.671952 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a34f97-2175-4bb6-af9d-ab5ece7492c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.720204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-27a34f97-2175-4bb6-af9d-ab5ece7492c7 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:28:29.721412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-27a34f97-2175-4bb6-af9d-ab5ece7492c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:29.728036 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1390/2761] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:28:29.740183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1fde-d380-4532-9b63-862d72d29c88 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:29.752038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b1fde-d380-4532-9b63-862d72d29c88 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:29.755536 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:29.756193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.756473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.756747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.764428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:29.765868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34db6-a6b6-42e8-b522-3828225bfc77 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:29.773606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1372/2762] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:29.858901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a773-547e-4673-84f0-c42f8c06c174 None None] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.859189 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a773-547e-4673-84f0-c42f8c06c174 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.859467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a773-547e-4673-84f0-c42f8c06c174 None None] PATH_INFO: `/` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.860855 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1391/2763] 10.222.0.23 () {54 vars in 921 bytes} [Fri Jun 24 15:28:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 15:28:29.865668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.865953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.866227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.903279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] MFA Rules not processed for user `4bfc33870df145a4ba481afd9cdab92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:28:29.904439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:28:29.913838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1373/2764] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:28:29.924268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a773-547e-4673-84f0-c42f8c06c174 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:28:29.936001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a773-547e-4673-84f0-c42f8c06c174 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:28:29.939627 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:28:29.940256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:28:29.940541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:28:29.940817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:28:29.952087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:28:29.953519 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:28:29.964197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1392/2765] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:28:39.900286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=15 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:28:39.959567 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=19 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:27.097768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:27.108249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6eb8e-c175-4430-9272-447f2abc8b55 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:27.112998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:27.113636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:27.113918 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:27.114193 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:27.116830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:27.177197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:27.178677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:27.188191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1374/2766] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:30.753170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:30.755871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:30.771945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29697722-3a42-43b1-a6d6-27c360fea9a3 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:30.778362 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:30.779026 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:30.779308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:30.779583 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:30.781304 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:30.793976 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:30.795582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:30.805926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1393/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:33.473375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:33.482379 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d682f72-6c0a-41b2-97df-887a0bc51588 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:33.486143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:33.487162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:33.487440 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:33.487740 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:33.489880 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:33.559932 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:33.561414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:33.571236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1375/2768] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:37.126073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:37.127524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:37.142112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d35bee-e05d-4f82-89bd-3e1bb2c342d8 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:37.147292 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:37.147934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:37.148244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:37.148541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:37.150251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:37.162933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:37.164534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:37.176638 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1394/2769] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:40.851901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:40.864306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcfa9-aff8-4e90-8cf4-40b4b7e1135a service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:40.869493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:40.870142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:40.870427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:40.870732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:40.872522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:40.884484 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:40.886174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:40.896692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1376/2770] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:30:40.968131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:40.969670 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:40.984763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215e1ce-ef59-40bc-bba8-14fcdc458b5d service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:40.989370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:40.990237 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:40.990550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:40.990827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] PATH_INFO: `/v3/users/f7793fa995e64744b2e5ee5cf1a91514` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:40.992007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7793fa995e64744b2e5ee5cf1a91514)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:40.993917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:41.025340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:41.027452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:41.106336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74c68-aa00-450e-bc57-34f7098c5802 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:41.109897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:41.110567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:41.110850 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:41.111125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:41.113011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:41.127660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:41.187004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7793fa995e64744b2e5ee5cf1a91514'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.197199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:41.199347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:41.199727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7793fa995e64744b2e5ee5cf1a91514'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.218488 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1377/2771] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:30:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:30:41.253163 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7793fa995e64744b2e5ee5cf1a91514'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.261321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7793fa995e64744b2e5ee5cf1a91514'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.262005 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7793fa995e64744b2e5ee5cf1a91514'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.268037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1395/2772] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:30:40 2022] DELETE /identity/v3/users/f7793fa995e64744b2e5ee5cf1a91514 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:41.534913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:41.551090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd07c-101e-48e2-812a-58313928e018 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:41.608063 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:41.608913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:41.609197 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:41.609472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] PATH_INFO: `/v3/projects/6a1633d3a76f4a54876b11fdba57636c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:41.610698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1633d3a76f4a54876b11fdba57636c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:41.612623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:41.722431 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1633d3a76f4a54876b11fdba57636c', 'request_id': 'req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.795890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1633d3a76f4a54876b11fdba57636c', 'request_id': 'req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.802342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1633d3a76f4a54876b11fdba57636c', 'request_id': 'req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.814606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] The token cache is being invalidate because project 6a1633d3a76f4a54876b11fdba57636c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:30:41.814896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.815547 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:41.817166 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1378/2773] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:30:41 2022] DELETE /identity/v3/projects/6a1633d3a76f4a54876b11fdba57636c => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:41.821394 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:41.823654 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:41.836846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529dffdd-50ef-4807-88ce-c8b665bef6cd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:41.893410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:41.894333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:41.894650 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:41.894932 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] PATH_INFO: `/v3/users/deb743191e1b42b68157737319bd1c5f` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:41.896149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb743191e1b42b68157737319bd1c5f)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:41.898081 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:42.094798 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb743191e1b42b68157737319bd1c5f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.095911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:42.108174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb743191e1b42b68157737319bd1c5f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.178524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb743191e1b42b68157737319bd1c5f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.186365 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb743191e1b42b68157737319bd1c5f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.187018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb743191e1b42b68157737319bd1c5f'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.192904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1396/2774] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:30:41 2022] DELETE /identity/v3/users/deb743191e1b42b68157737319bd1c5f => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:42.505956 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:30:42.507631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:30:42.520362 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa86a4-6f8a-4e08-80ba-e74eb5d9b006 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:30:42.540699 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:42.587651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:30:42.589612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:30:42.589901 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:30:42.590184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] PATH_INFO: `/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:30:42.591792 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2c0d736c5f4236b9b1ef7077a1f331)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:30:42.593765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:30:42.732169 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2c0d736c5f4236b9b1ef7077a1f331', 'request_id': 'req-f2c67545-d307-4f9d-a00f-7baa2b99a58e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.813248 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2c0d736c5f4236b9b1ef7077a1f331', 'request_id': 'req-f2c67545-d307-4f9d-a00f-7baa2b99a58e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.820062 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2c0d736c5f4236b9b1ef7077a1f331', 'request_id': 'req-f2c67545-d307-4f9d-a00f-7baa2b99a58e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.831774 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] The token cache is being invalidate because project 6b2c0d736c5f4236b9b1ef7077a1f331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:30:42.832061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.832705 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:30:42.834244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1379/2775] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:30:42 2022] DELETE /identity/v3/projects/6b2c0d736c5f4236b9b1ef7077a1f331 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:30:52.350582 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:30:52.828257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:05.564863 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:05.566361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:05.635293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31bd7c-460e-48ee-9fbd-16f0a0871372 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:05.697728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:05.698397 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:05.698716 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:05.698997 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:05.700812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:05.713475 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:05.715120 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:05.726264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1397/2776] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:33:05.953807 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:05.962667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:06.023363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c67545-d307-4f9d-a00f-7baa2b99a58e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:06.026947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:06.027590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:06.027870 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:06.028151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:06.029962 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:06.092944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:06.094392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:06.104188 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1380/2777] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:33:09.709879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:09.711449 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:09.725778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405c9be-7353-4bd9-aff5-a581936186a1 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:09.730358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:09.731256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:09.731539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:09.731810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] PATH_INFO: `/v3/users/ea0f85ae8bd942338bfcda177b6864a6` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:09.732978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0f85ae8bd942338bfcda177b6864a6)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:09.734919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:09.958666 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0f85ae8bd942338bfcda177b6864a6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:09.970201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0f85ae8bd942338bfcda177b6864a6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.036226 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0f85ae8bd942338bfcda177b6864a6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.043613 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0f85ae8bd942338bfcda177b6864a6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.044236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-9a3c2636-afe5-46aa-b95c-b6a25eba0140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0f85ae8bd942338bfcda177b6864a6'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.050599 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1398/2778] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:33:09 2022] DELETE /identity/v3/users/ea0f85ae8bd942338bfcda177b6864a6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:10.292509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:10.310944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9273321-f864-44d3-9643-e069a07d93d5 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:10.379129 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:10.380148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.380430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.380709 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] PATH_INFO: `/v3/projects/132c0bbdee9e46388793b33cda9ee10c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.382433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=132c0bbdee9e46388793b33cda9ee10c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:10.383849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:10.486199 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:10.490505 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '132c0bbdee9e46388793b33cda9ee10c', 'request_id': 'req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.559857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '132c0bbdee9e46388793b33cda9ee10c', 'request_id': 'req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.566500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '132c0bbdee9e46388793b33cda9ee10c', 'request_id': 'req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.576718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] The token cache is being invalidate because project 132c0bbdee9e46388793b33cda9ee10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:33:10.577000 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.577620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d1a7e45e-c30b-4064-b8a8-3d30deb0fa97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:10.579349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1381/2779] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:33:10 2022] DELETE /identity/v3/projects/132c0bbdee9e46388793b33cda9ee10c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:33:10.585122 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.585407 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.585705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.592971 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:10.603979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:10.605174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:10.662657 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1399/2780] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:33:10.670479 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.670796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.671076 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.689782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:10.691090 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:10.700084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1382/2781] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:10.704357 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:10.714448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71e8247-a2fb-4eba-9a55-8c8da8ad7d2d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:10.718993 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:10.719637 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.719918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.720193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.720912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:33:10.721386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:10.723053 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:10.738621 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1400/2782] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:33:10.739804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:10.749261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaa4f6-eed4-4345-989b-2849bf61c231 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:10.753876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:10.754568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.754883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.755130 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.755820 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:10.757727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:10.841861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1383/2783] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:33:10 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:33:10.845620 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:10.857369 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3b2b6-1938-4b63-8186-251dfdecc841 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:10.912485 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:10.913144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:10.913429 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:10.913704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:10.914389 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:10.916328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:11.016686 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1401/2784] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:33:10 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:33:11.020965 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:11.031089 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0c37b-8e61-46ca-bfb6-0a7a704fcbee admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:11.036520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:11.038421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.038987 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.039267 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.039998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:11.041590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:11.053008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1384/2785] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:11.057056 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:11.067051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac47a0-3268-467a-9138-a6b48c66a8a5 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:11.071823 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:11.072470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.072750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.073022 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.073740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:11.075384 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729231f-e8e8-426a-865c-87c737bc638b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:11.085090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1402/2786] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:11.089080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:11.099647 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9761c2-4f0d-4d59-b80e-058df0012e26 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:11.104244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:11.105113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.105391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.105662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] PATH_INFO: `/v3/projects/8e73af1c054f473484fe3c910e7d5a43/users/4931005b98884388bb68b6462c01867e/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.120668 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e73af1c054f473484fe3c910e7d5a43, user_id=4931005b98884388bb68b6462c01867e, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:11.123397 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14845a0-1d56-48d9-ae45-77c9e7fc1428 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:11.228549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1385/2787] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:11 2022] PUT /identity/v3/projects/8e73af1c054f473484fe3c910e7d5a43/users/4931005b98884388bb68b6462c01867e/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:11.233784 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.234090 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.234375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.252717 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:11.253930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:11.312705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1403/2788] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:11.322011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227040fe-a00d-455d-b5fc-8540e3539ce7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.322477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227040fe-a00d-455d-b5fc-8540e3539ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.322628 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227040fe-a00d-455d-b5fc-8540e3539ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.341762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-227040fe-a00d-455d-b5fc-8540e3539ce7 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:11.343016 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-227040fe-a00d-455d-b5fc-8540e3539ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:11.394720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1386/2789] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Jun 24 15:33:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:11.401355 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:11.410541 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab13578-547c-4e66-be19-c2cb8001a0a1 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:11.414923 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:11.415585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:11.415907 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:11.416183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:11.425354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:11.427082 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:11.430908 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:11.436896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1404/2790] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:33:12.728938 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:12.729455 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:12.729611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:12.757269 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] MFA Rules not processed for user `4931005b98884388bb68b6462c01867e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:12.758620 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:12.828118 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1387/2791] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:12.832386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:12.843585 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a5cbe-394c-4bb4-800c-052bf0bd0fb0 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:12.848265 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:12.849442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:12.852523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:12.852523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:12.852523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:12.852799 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:12.929892 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1405/2792] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:33:12 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:33:12.933925 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:12.945009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca811d-1bd2-4e47-a5e1-f07a23df0fe3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:12.948655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:13.002529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:13.003237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:13.003520 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:13.003796 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:13.004497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:13.006420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:13.099302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1388/2793] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:33:12 2022] POST /identity/v3/users => generated 309 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:33:13.100667 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:13.112188 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066bb1c-df32-4b2a-9045-c67e51044c3b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:13.117035 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:13.117688 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:13.117966 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:13.118238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:13.122694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:13.122694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:13.131512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1406/2794] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:13.135593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:13.146098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec340c20-6c00-441f-8701-f2091772c0aa admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:13.150706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:13.151580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:13.151859 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:13.152133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] PATH_INFO: `/v3/projects/b0f609c172c14bb9909937059bf8d0ba/users/20bb4dfdddec4cabb67c6a5d55fdf81f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:13.167480 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f609c172c14bb9909937059bf8d0ba, user_id=20bb4dfdddec4cabb67c6a5d55fdf81f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:13.170443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:13.288648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1389/2795] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:13 2022] PUT /identity/v3/projects/b0f609c172c14bb9909937059bf8d0ba/users/20bb4dfdddec4cabb67c6a5d55fdf81f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:13.292499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:13.302960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec91532-33c0-4127-a513-e481275a89d3 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:13.358933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:13.359610 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:13.359895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:13.360174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:13.360901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:13.362534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccbcf6-472c-4e6b-8c8a-eaf65373a21b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:13.372895 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1407/2796] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:13.377115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:13.388402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f8817-8adf-4853-9a30-ac38cbbc1eac admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:13.393074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:13.393942 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:13.394221 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:13.394490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] PATH_INFO: `/v3/projects/b0f609c172c14bb9909937059bf8d0ba/users/20bb4dfdddec4cabb67c6a5d55fdf81f/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:13.397241 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f609c172c14bb9909937059bf8d0ba, user_id=20bb4dfdddec4cabb67c6a5d55fdf81f, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:13.399900 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:13.410113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1390/2797] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:13 2022] PUT /identity/v3/projects/b0f609c172c14bb9909937059bf8d0ba/users/20bb4dfdddec4cabb67c6a5d55fdf81f/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:14.702049 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:14.702593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:14.702593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:14.733114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] MFA Rules not processed for user `20bb4dfdddec4cabb67c6a5d55fdf81f`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:14.734455 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:14.807255 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1408/2798] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:16.132085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:16.199649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190e968-f6df-47d2-a114-9072a43c56c1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:16.204048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:16.204902 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:16.205180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:16.205452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] PATH_INFO: `/v3/users/4931005b98884388bb68b6462c01867e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:16.206651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4931005b98884388bb68b6462c01867e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:16.208698 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:16.402038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4931005b98884388bb68b6462c01867e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:16.412898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:16.414899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4931005b98884388bb68b6462c01867e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:16.491407 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4931005b98884388bb68b6462c01867e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:16.498688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4931005b98884388bb68b6462c01867e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:16.499350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4931005b98884388bb68b6462c01867e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:16.507475 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1391/2799] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:33:16 2022] DELETE /identity/v3/users/4931005b98884388bb68b6462c01867e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:16.818824 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:16.831470 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570adb6d-276b-401a-b9bd-a7b08c980121 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:16.876904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:16.892554 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:16.893409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:16.893689 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:16.893963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] PATH_INFO: `/v3/projects/8e73af1c054f473484fe3c910e7d5a43` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:16.895202 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e73af1c054f473484fe3c910e7d5a43)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:16.897763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:17.043813 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e73af1c054f473484fe3c910e7d5a43', 'request_id': 'req-46c4d982-a5ff-4b6a-88a8-909015bc73ee'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.050401 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e73af1c054f473484fe3c910e7d5a43', 'request_id': 'req-46c4d982-a5ff-4b6a-88a8-909015bc73ee'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.113889 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e73af1c054f473484fe3c910e7d5a43', 'request_id': 'req-46c4d982-a5ff-4b6a-88a8-909015bc73ee'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.124543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] The token cache is being invalidate because project 8e73af1c054f473484fe3c910e7d5a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:33:17.124852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.125469 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.127042 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1409/2800] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:33:16 2022] DELETE /identity/v3/projects/8e73af1c054f473484fe3c910e7d5a43 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:33:17.131860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:17.133260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:17.147045 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcda07-ffee-49c7-9fac-a792a803d066 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:17.203906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:17.204781 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:17.205066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:17.205343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] PATH_INFO: `/v3/users/20bb4dfdddec4cabb67c6a5d55fdf81f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:17.206546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] RBAC: Authorizing `identity:delete_user(user_id=20bb4dfdddec4cabb67c6a5d55fdf81f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:17.209120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:17.404765 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20bb4dfdddec4cabb67c6a5d55fdf81f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.417361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20bb4dfdddec4cabb67c6a5d55fdf81f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.482727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20bb4dfdddec4cabb67c6a5d55fdf81f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.491027 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20bb4dfdddec4cabb67c6a5d55fdf81f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.491655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a114af0-8b14-47f6-8d0a-9f1c7208281c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20bb4dfdddec4cabb67c6a5d55fdf81f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.497642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1392/2801] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:33:17 2022] DELETE /identity/v3/users/20bb4dfdddec4cabb67c6a5d55fdf81f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:17.795721 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:17.797441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:17.811003 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4d982-a5ff-4b6a-88a8-909015bc73ee admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:17.870641 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:17.871440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:17.871723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:17.872001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] PATH_INFO: `/v3/projects/b0f609c172c14bb9909937059bf8d0ba` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:17.873245 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f609c172c14bb9909937059bf8d0ba)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:17.875232 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:17.990013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f609c172c14bb9909937059bf8d0ba', 'request_id': 'req-74cfad77-fd5a-4693-b1d2-114e5deaf277'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:17.996316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f609c172c14bb9909937059bf8d0ba', 'request_id': 'req-74cfad77-fd5a-4693-b1d2-114e5deaf277'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:18.067917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f609c172c14bb9909937059bf8d0ba', 'request_id': 'req-74cfad77-fd5a-4693-b1d2-114e5deaf277'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:18.079134 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] The token cache is being invalidate because project b0f609c172c14bb9909937059bf8d0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:33:18.079415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:18.080027 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-74cfad77-fd5a-4693-b1d2-114e5deaf277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:33:18.081565 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1410/2802] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:33:17 2022] DELETE /identity/v3/projects/b0f609c172c14bb9909937059bf8d0ba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:18.086623 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.086914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.087196 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.106042 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:18.107367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:18.157528 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:18.166958 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1393/2803] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:18.174910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.175201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.175480 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.188129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:18.194200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:18.195457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:18.203535 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1411/2804] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:18.207736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.220215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a9b2f-90c4-463f-8914-2a7d6e4aba8b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.225823 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.226473 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.226789 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.227067 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.227787 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:33:18.228113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.229711 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.241277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1394/2805] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:33:18.245777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.255166 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf32dc56-629e-4480-9631-49f8f98d3e05 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.260137 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.260779 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.261058 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.261329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.262001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.264270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.379435 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1412/2806] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:33:18.383695 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.393833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32743f4e-c713-4cb7-8c1e-97ccf475f3ef admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.451228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.451821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.452100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.452371 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.453092 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.455043 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.536673 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1395/2807] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/users => generated 289 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:33:18.540837 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.551420 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428973ba-9a5d-4fd2-8598-5cfb1110fe5e admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.556321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.556980 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.557263 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.557539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.558288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.559930 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.570472 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1413/2808] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:18.574655 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.585244 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e07a2b-6935-4cec-a624-7e365f4f3d2d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.590168 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.590853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.591206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.591477 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.592200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.593892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d989775-b64b-4823-8546-32ce1692e4fe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.604146 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1396/2809] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:18.608161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.618047 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44716d9-3376-461c-a598-0cfc984aaf0d admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.622626 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.623509 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.623788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.624065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] PATH_INFO: `/v3/projects/79eb33414e7d45679cedcdd89302d6df/users/19fb534760674175a4cfb3be42ebed86/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.638983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79eb33414e7d45679cedcdd89302d6df, user_id=19fb534760674175a4cfb3be42ebed86, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.641625 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:18.780203 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1414/2810] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:18 2022] PUT /identity/v3/projects/79eb33414e7d45679cedcdd89302d6df/users/19fb534760674175a4cfb3be42ebed86/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:18.790764 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.794801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.794801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.818080 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] MFA Rules not processed for user `19fb534760674175a4cfb3be42ebed86`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:18.819429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:18.883606 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1397/2811] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:33:18.888062 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:18.952260 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98524dde-5faf-49c3-94df-53ad3601af6c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:18.956888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:18.957539 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:18.957848 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:18.958173 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:18.958957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:18.960858 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.043949 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1415/2812] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:33:18 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:33:19.048238 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.059923 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae57e6-24c0-41f6-a060-eba11adf095c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.108445 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:19.117889 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.118584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.118876 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.121036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.121036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.121884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.232215 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1398/2813] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:33:19 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:33:19.236343 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.247734 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a357a-6037-4d7b-a566-11082ca847da admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.252405 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.253097 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.253382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.253662 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.254391 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.256238 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.267114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1416/2814] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:19.272944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.283183 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883aee-9f8a-4f5f-82bd-8a7d10c0bd8a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.288920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.289861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.290351 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.290857 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] PATH_INFO: `/v3/projects/d6451802ce3145a6803a16988cd62628/users/8f84c11db83841c5928709b165b9560d/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.309572 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6451802ce3145a6803a16988cd62628, user_id=8f84c11db83841c5928709b165b9560d, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.312302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.391993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1399/2815] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:19 2022] PUT /identity/v3/projects/d6451802ce3145a6803a16988cd62628/users/8f84c11db83841c5928709b165b9560d/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:19.395998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.405977 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b2c0f-c269-42f7-a661-57199a141834 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.461838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.462496 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.462811 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.463109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.463826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.465425 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d87169-f62f-4170-abab-5e392cbcfba8 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.475548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1417/2816] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:33:19.479675 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.491048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8569f034-f4ea-43a7-bf4a-825a0ae65f5a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.495940 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.496814 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.497093 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.497361 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] PATH_INFO: `/v3/projects/d6451802ce3145a6803a16988cd62628/users/8f84c11db83841c5928709b165b9560d/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.499372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6451802ce3145a6803a16988cd62628, user_id=8f84c11db83841c5928709b165b9560d, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.502028 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.512836 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1400/2817] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:33:19 2022] PUT /identity/v3/projects/d6451802ce3145a6803a16988cd62628/users/8f84c11db83841c5928709b165b9560d/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:33:19.569548 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.569750 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.570023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.594298 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:19.597345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] MFA Rules not processed for user `8f84c11db83841c5928709b165b9560d`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:33:19.599112 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:19.667562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1418/2818] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:33:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:33:19.678590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:19.680402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:33:19.737855 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c40ff3-a748-4b11-9d6e-5c2914499e2a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:19.741708 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:19.742344 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:19.742662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:19.742935 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:19.798549 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:19.799989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:19.808895 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1401/2819] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:33:29.380440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:33:29.390998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efad514-bff1-4b4d-af9b-122b7c5f6586 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:33:29.394596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:33:29.395236 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:33:29.395517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:33:29.395792 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:33:29.404943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:33:29.406386 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c45b58-6bbe-43c0-a5ec-77cac7a658ae service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:33:29.416370 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1419/2820] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:33:29 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:33:29.804808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:33:39.412020 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=34 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:04.723667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:04.788157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fba243-1114-4cef-a96a-7dc28e6597a5 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:04.792676 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:04.793543 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:04.793821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:04.794095 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] PATH_INFO: `/v3/users/19fb534760674175a4cfb3be42ebed86` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:04.795291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19fb534760674175a4cfb3be42ebed86)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:04.797201 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:04.985715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19fb534760674175a4cfb3be42ebed86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:04.997493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19fb534760674175a4cfb3be42ebed86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:05.066920 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19fb534760674175a4cfb3be42ebed86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:05.074125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19fb534760674175a4cfb3be42ebed86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:05.074777 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19fb534760674175a4cfb3be42ebed86'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:05.081218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1402/2821] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:04 2022] DELETE /identity/v3/users/19fb534760674175a4cfb3be42ebed86 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:05.085853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:05.089084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:05.089084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:05.106083 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:05.107332 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:05.166851 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1420/2822] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:05.177159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:05.178642 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:05.234281 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee661-be81-4239-af1a-05edd4857735 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:05.238762 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:05.239346 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:05.239631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:05.239907 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:05.249124 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:05.250580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:05.261198 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1403/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:05.796336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:05.798275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:05.813812 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8a344-0ba9-41c4-9988-3b84a69fbb69 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:05.820320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:05.821241 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:05.821448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:05.821722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] PATH_INFO: `/v3/projects/79eb33414e7d45679cedcdd89302d6df` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:05.823034 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=79eb33414e7d45679cedcdd89302d6df)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:05.824956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:05.955233 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79eb33414e7d45679cedcdd89302d6df', 'request_id': 'req-97281de8-bebd-45ab-b104-a6a68a8838ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:05.961516 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79eb33414e7d45679cedcdd89302d6df', 'request_id': 'req-97281de8-bebd-45ab-b104-a6a68a8838ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.022879 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79eb33414e7d45679cedcdd89302d6df', 'request_id': 'req-97281de8-bebd-45ab-b104-a6a68a8838ce'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.033624 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] The token cache is being invalidate because project 79eb33414e7d45679cedcdd89302d6df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:06.033910 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.034551 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.036333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1421/2824] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:05 2022] DELETE /identity/v3/projects/79eb33414e7d45679cedcdd89302d6df => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:34:06.040209 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:06.049906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb1a9f-354f-40d6-b6ae-f3984bd05d27 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:06.069257 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:06.108315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:06.109171 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:06.109447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:06.109715 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] PATH_INFO: `/v3/users/8f84c11db83841c5928709b165b9560d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:06.110924 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f84c11db83841c5928709b165b9560d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:06.112824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:06.281524 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f84c11db83841c5928709b165b9560d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.294053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f84c11db83841c5928709b165b9560d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.358497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f84c11db83841c5928709b165b9560d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.367373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f84c11db83841c5928709b165b9560d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.368031 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-a4fdcf91-dc2f-4313-acb0-c1a2e4edf150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f84c11db83841c5928709b165b9560d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.374120 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1404/2825] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:06 2022] DELETE /identity/v3/users/8f84c11db83841c5928709b165b9560d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:06.791297 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:06.797419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:06.807416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97281de8-bebd-45ab-b104-a6a68a8838ce admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:06.866155 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:06.866975 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:06.867259 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:06.867536 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] PATH_INFO: `/v3/projects/d6451802ce3145a6803a16988cd62628` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:06.868724 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6451802ce3145a6803a16988cd62628)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:06.870669 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:06.956092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:06.973244 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6451802ce3145a6803a16988cd62628', 'request_id': 'req-61e919a8-25f6-42be-9d14-fcefa806e63e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:06.979903 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6451802ce3145a6803a16988cd62628', 'request_id': 'req-61e919a8-25f6-42be-9d14-fcefa806e63e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:07.046178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6451802ce3145a6803a16988cd62628', 'request_id': 'req-61e919a8-25f6-42be-9d14-fcefa806e63e'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:07.058336 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] The token cache is being invalidate because project d6451802ce3145a6803a16988cd62628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80353) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:07.058672 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:07.059295 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-61e919a8-25f6-42be-9d14-fcefa806e63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:07.060918 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1422/2826] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:06 2022] DELETE /identity/v3/projects/d6451802ce3145a6803a16988cd62628 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:07.065843 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.066319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.066414 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.086603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:07.087815 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:07.147539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1405/2827] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:07.155344 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.155636 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.155931 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.178446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:07.179753 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:07.188460 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1423/2828] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:07.192893 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.202363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7f67f-4e14-482a-bc01-9a06e9605bc8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.206890 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.207575 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.208518 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.208799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.209546 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:34:07.209854 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.211612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.222858 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1406/2829] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:34:07.227011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.236154 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d1df23-c689-4b01-b3bf-930cf8592374 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.241015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.241660 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.241942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.242218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.242939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.244839 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.354700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1424/2830] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:07.358933 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.369913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca391789-fc77-475e-8cc4-d6115749cd53 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.396743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:07.425922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.427254 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.427604 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.428103 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.429236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.431256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.523878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1407/2831] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:34:07.527917 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.539239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e44ded-b460-459b-b07d-a55b874ecb27 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.544040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.544701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.544985 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.545264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.546051 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.547694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.559527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1425/2832] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:07.563296 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.574597 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39488d97-90c7-476c-8a4a-cb3f62139c4a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.579227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.579908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.580191 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.580467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.581184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.582959 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63a856-ec42-4330-b034-b32fe5c2ea7e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.592826 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1408/2833] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:07.596739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.606843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850a908-6a33-4068-8bdd-ed752e7072eb admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.611531 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.612441 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.612723 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.612995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] PATH_INFO: `/v3/projects/19fbf874141d4d648a87fe825ea8da3e/users/e5a38af0752c4fec9fed8c5cca0b93a3/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.628147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19fbf874141d4d648a87fe825ea8da3e, user_id=e5a38af0752c4fec9fed8c5cca0b93a3, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.631600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:07.712665 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1426/2834] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:07 2022] PUT /identity/v3/projects/19fbf874141d4d648a87fe825ea8da3e/users/e5a38af0752c4fec9fed8c5cca0b93a3/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:07.724236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.724522 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.724825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.751452 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] MFA Rules not processed for user `e5a38af0752c4fec9fed8c5cca0b93a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:07.752670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:07.816561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1409/2835] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:07.821270 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:07.881990 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fba69-7bba-4eb3-ba95-fa5f0b758056 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:07.886648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:07.887293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:07.887571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:07.887843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:07.888527 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:07.890412 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.029179 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1427/2836] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:07 2022] POST /identity/v3/projects => generated 387 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:08.033338 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.044594 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b21a7-e7a1-4715-a983-2b4058176b43 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.101651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.102315 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.102629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.102910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.103592 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.105498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.198717 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1410/2837] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:08 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:08.202845 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.213649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672654a-db5f-4dfe-ae27-21e9ce26b7e2 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.218342 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.219032 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.219322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.219605 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.220321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.221915 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.233271 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1428/2838] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 15:34:08.237380 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.247553 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ff556-48c4-4313-8c1a-531de0aac8bc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.252159 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.253061 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.253340 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.253615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] PATH_INFO: `/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123/users/702a6a4e370a45aca0f630e3eb8c43c2/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.269415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a01a32ea6347d0bd11b75eca0b4123, user_id=702a6a4e370a45aca0f630e3eb8c43c2, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.272896 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.346867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1411/2839] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:08 2022] PUT /identity/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123/users/702a6a4e370a45aca0f630e3eb8c43c2/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:08.351513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.361648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd56013-fea4-4495-9602-9cac0ece35b9 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.382009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:08.418868 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.419537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.419888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.420318 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.421213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.423553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3aafa-e537-434d-9016-3adaf96a5b67 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.433282 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1429/2840] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:08.437605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.443849 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:08.449648 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db5ef5-9761-417c-8cc5-79cbb164e71d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.454347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.455286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.455567 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.460142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] PATH_INFO: `/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123/users/702a6a4e370a45aca0f630e3eb8c43c2/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.460142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a01a32ea6347d0bd11b75eca0b4123, user_id=702a6a4e370a45aca0f630e3eb8c43c2, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.461114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb80bb1-c026-43af-b63f-0e78264086fe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.471290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1412/2841] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:08 2022] PUT /identity/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123/users/702a6a4e370a45aca0f630e3eb8c43c2/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:08.481801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.482107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.482421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.508748 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] MFA Rules not processed for user `702a6a4e370a45aca0f630e3eb8c43c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:08.509956 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:08.580634 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1430/2842] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:08.592154 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.592444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.592719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.630149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] MFA Rules not processed for user `c42ba8788bf2431a96b2087b287d7426`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:08.631290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:08.685432 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1413/2843] 10.222.0.23 () {58 vars in 1060 bytes} [Fri Jun 24 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:08.691168 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.701023 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f610855-15d3-4373-b579-6215ba6c3be7 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.704593 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.705242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.705524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.705816 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.715553 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.716984 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4d572-3b60-4b56-b78f-78adca6673e8 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:08.726500 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1431/2844] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:08.792875 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:08.855144 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e60182-d998-48ea-b4c6-b8a32da4afef None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:08.859701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:08.860562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:08.860838 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:08.861113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] PATH_INFO: `/v3/users/e5a38af0752c4fec9fed8c5cca0b93a3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:08.862286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a38af0752c4fec9fed8c5cca0b93a3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:08.864754 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:09.099568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a38af0752c4fec9fed8c5cca0b93a3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:09.111143 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a38af0752c4fec9fed8c5cca0b93a3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:09.188898 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a38af0752c4fec9fed8c5cca0b93a3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:09.196828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a38af0752c4fec9fed8c5cca0b93a3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:09.197463 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3c316e9c-270e-40ec-9031-9d969b282ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a38af0752c4fec9fed8c5cca0b93a3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:09.203465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1414/2845] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:08 2022] DELETE /identity/v3/users/e5a38af0752c4fec9fed8c5cca0b93a3 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:09.208119 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:09.208691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:09.208691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:09.236898 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:09.238104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:09.308710 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1432/2846] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:09.317927 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:09.318212 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:09.318485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:09.325631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:09.336492 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] MFA Rules not processed for user `d08dcd615d87450d9ccdc58fe89f4014`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:09.337677 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:09.388571 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1415/2847] 10.222.0.23 () {58 vars in 1061 bytes} [Fri Jun 24 15:34:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:09.394147 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:09.403735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44ec623-7e19-42ce-bcdf-7a249ce33e5d None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:09.407520 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:09.408149 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:09.408427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:09.408701 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:09.418645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:09.420104 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:09.430273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1433/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:09.852211 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:09.853636 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:09.883200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d46dd1-728e-4ac1-87ef-617a4e02a235 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:09.887860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:09.888724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:09.889008 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:09.889304 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] PATH_INFO: `/v3/projects/19fbf874141d4d648a87fe825ea8da3e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:09.890493 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19fbf874141d4d648a87fe825ea8da3e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:09.892497 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:10.029952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19fbf874141d4d648a87fe825ea8da3e', 'request_id': 'req-928756be-555c-4914-bcf1-17578e5bd447'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.102947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19fbf874141d4d648a87fe825ea8da3e', 'request_id': 'req-928756be-555c-4914-bcf1-17578e5bd447'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.109461 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19fbf874141d4d648a87fe825ea8da3e', 'request_id': 'req-928756be-555c-4914-bcf1-17578e5bd447'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.120387 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] The token cache is being invalidate because project 19fbf874141d4d648a87fe825ea8da3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:10.120955 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.121307 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.122863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1416/2849] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:09 2022] DELETE /identity/v3/projects/19fbf874141d4d648a87fe825ea8da3e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:10.127084 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:10.132474 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=100 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:10.137586 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0161d8-b885-44f7-be73-b4041d5f7638 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:10.192865 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:10.194213 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:10.194494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:10.194801 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] PATH_INFO: `/v3/users/702a6a4e370a45aca0f630e3eb8c43c2` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:10.195973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a6a4e370a45aca0f630e3eb8c43c2)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:10.197919 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:10.354661 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a6a4e370a45aca0f630e3eb8c43c2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.367537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a6a4e370a45aca0f630e3eb8c43c2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.480451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a6a4e370a45aca0f630e3eb8c43c2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.488070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a6a4e370a45aca0f630e3eb8c43c2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.488708 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-e00fb6b5-1d78-4ab7-aa57-9e0067231881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a6a4e370a45aca0f630e3eb8c43c2'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:10.495125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1434/2850] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:10 2022] DELETE /identity/v3/users/702a6a4e370a45aca0f630e3eb8c43c2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:10.979339 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:10.980887 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:11.000657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928756be-555c-4914-bcf1-17578e5bd447 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.055020 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.055894 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.056182 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.056459 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] PATH_INFO: `/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.057704 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a01a32ea6347d0bd11b75eca0b4123)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.059660 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.127973 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:11.142743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a01a32ea6347d0bd11b75eca0b4123', 'request_id': 'req-12535821-e04b-47bd-9d42-335722ebe57a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:11.188413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a01a32ea6347d0bd11b75eca0b4123', 'request_id': 'req-12535821-e04b-47bd-9d42-335722ebe57a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:11.194919 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a01a32ea6347d0bd11b75eca0b4123', 'request_id': 'req-12535821-e04b-47bd-9d42-335722ebe57a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:11.205799 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] The token cache is being invalidate because project f3a01a32ea6347d0bd11b75eca0b4123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:11.206087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:11.206753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-12535821-e04b-47bd-9d42-335722ebe57a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:11.208401 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1417/2851] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:10 2022] DELETE /identity/v3/projects/f3a01a32ea6347d0bd11b75eca0b4123 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:11.213934 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.214222 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.214499 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.241924 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:11.243115 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:11.303538 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1435/2852] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:11.311245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.311540 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.311821 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.329804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:11.331038 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:11.339544 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1418/2853] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:11.343648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.354506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf3af-33aa-479c-914d-2cf49484a5c3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.359111 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.359769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.360048 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.360320 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.361043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:34:11.361350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.363013 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.375810 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1436/2854] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:34:11.380087 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.390117 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a0416e-da4e-4262-af21-1f8533d54072 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.394772 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.395422 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.395706 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.395983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.397427 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.399363 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.499321 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1419/2855] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:11 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:11.503406 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.509356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:11.514826 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d07237-5b23-45ff-9678-208373851e24 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.571206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.571867 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.572157 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.572434 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.573125 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.575092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.692133 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1437/2856] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:11 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:11.698494 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.709577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7a0db-5d8f-4f38-aac9-cc00e025ed4d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.714152 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.714853 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.715138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.715408 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.716128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.717773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.728884 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1420/2857] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:11.732818 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.743375 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fa6b2-30b7-4537-8333-73f584a58a2c admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.748004 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.748680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.748958 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.749234 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.749996 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.751571 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49852dd9-4c04-4139-9b74-4b1ff0881d4f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.762121 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1438/2858] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:11.766816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.777486 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef7a3-de24-4aee-8ade-09faf8b04eb8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:11.782006 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:11.783133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.783415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.783691 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] PATH_INFO: `/v3/projects/3b1514963d92436eae7cf9617041b8dd/users/cc38022e006a47edb07187a84d21ef18/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.798910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1514963d92436eae7cf9617041b8dd, user_id=cc38022e006a47edb07187a84d21ef18, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:11.801556 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:11.880104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1421/2859] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:11 2022] PUT /identity/v3/projects/3b1514963d92436eae7cf9617041b8dd/users/cc38022e006a47edb07187a84d21ef18/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:11.885803 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69e80-a896-4f5b-8193-dd4f67c0c756 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:11.886201 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69e80-a896-4f5b-8193-dd4f67c0c756 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:11.886489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b69e80-a896-4f5b-8193-dd4f67c0c756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:11.904421 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c6b69e80-a896-4f5b-8193-dd4f67c0c756 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:11.905616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c6b69e80-a896-4f5b-8193-dd4f67c0c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:11.964261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1439/2860] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:11.973657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:11.975388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:12.046114 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95afac-13ca-48e0-8c7f-6e9711c67107 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:12.050088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:12.050766 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:12.051048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:12.051319 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:12.060639 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:12.062066 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:12.071476 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1422/2861] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:13.315974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.316743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.316743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.345114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] MFA Rules not processed for user `cc38022e006a47edb07187a84d21ef18`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:13.346337 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:13.415512 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1440/2862] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:13.420145 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.430743 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233daa3-643f-4e6b-babe-aa73f681c507 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.437738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.437738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.437738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.437738 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.438094 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:13.439148 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:13.538605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1423/2863] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:13 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:13.542703 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.554070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3b457-0f84-4680-accc-b08f65bb81f9 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.561843 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:13.611273 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.611945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.612229 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.612508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.613207 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:13.615742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:13.708978 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1441/2864] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:13 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:13.713205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.723469 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32fa5-a6f0-4a1f-a3df-47e76848752e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.730323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.731034 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.731323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.731600 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.732336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:13.733951 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:13.744873 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1424/2865] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:13.750268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.760616 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cdd362-46b6-4f3e-a4bd-97282073ed90 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.765361 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.766239 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.766543 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.766830 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] PATH_INFO: `/v3/projects/9301e01a38654f1ea8bc2290304a710c/users/32f349967335404bba261da2eb6f3cba/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.782835 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9301e01a38654f1ea8bc2290304a710c, user_id=32f349967335404bba261da2eb6f3cba, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:13.785515 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:13.890489 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1442/2866] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:13 2022] PUT /identity/v3/projects/9301e01a38654f1ea8bc2290304a710c/users/32f349967335404bba261da2eb6f3cba/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:13.894368 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.900804 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:13.906247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1febca-ef2c-401d-8cdb-9e992f9ff1e2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.963941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.964659 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.964945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:13.965224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:13.965952 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:13.967584 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b6c5fb-d14d-4f97-a209-d604618434c3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:13.977773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1425/2867] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:13.981925 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:13.992283 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3045ef-f5c7-485a-b413-d04957771a8a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:13.996864 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:13.997751 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:13.998000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:14.003225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] PATH_INFO: `/v3/projects/9301e01a38654f1ea8bc2290304a710c/users/32f349967335404bba261da2eb6f3cba/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:14.003225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9301e01a38654f1ea8bc2290304a710c, user_id=32f349967335404bba261da2eb6f3cba, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:14.003225 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:14.013596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1443/2868] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:13 2022] PUT /identity/v3/projects/9301e01a38654f1ea8bc2290304a710c/users/32f349967335404bba261da2eb6f3cba/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:15.233096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:15.233096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:15.233096 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:15.258747 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] MFA Rules not processed for user `32f349967335404bba261da2eb6f3cba`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:15.259971 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:15.328614 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1426/2869] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:15.340127 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:15.341866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:15.399373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3234e-3b98-453b-9329-c7041310c9c7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:15.403021 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:15.403664 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:15.403945 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:15.404218 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:15.460490 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:15.461935 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:15.470834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1444/2870] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:17.192286 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:17.193751 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:17.206279 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0662f-c2c7-480e-9e77-80e4d3430aad None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:17.209886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:17.211247 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:17.211945 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:17.212251 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:17.221354 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:17.222816 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:17.233357 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1427/2871] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:17 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:25.466739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:27.229126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=68 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:34.465302 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:34.530987 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534839c-f6f0-4f8e-9439-9a431464ecf9 service cinder] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:34.535461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:34.536352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:34.536628 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:34.536899 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] PATH_INFO: `/v3/users/cc38022e006a47edb07187a84d21ef18` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:34.538064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc38022e006a47edb07187a84d21ef18)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:34.540007 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:34.820096 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc38022e006a47edb07187a84d21ef18'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:34.832769 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc38022e006a47edb07187a84d21ef18'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:34.977741 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc38022e006a47edb07187a84d21ef18'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:34.985629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc38022e006a47edb07187a84d21ef18'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:34.988871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc38022e006a47edb07187a84d21ef18'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:34.992400 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1445/2872] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:34 2022] DELETE /identity/v3/users/cc38022e006a47edb07187a84d21ef18 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:35.265350 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:35.278255 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b4a25-2620-447b-b74e-8dbdb95a6531 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:35.337726 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:35.338650 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:35.339013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:35.339289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] PATH_INFO: `/v3/projects/3b1514963d92436eae7cf9617041b8dd` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:35.340538 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1514963d92436eae7cf9617041b8dd)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:35.342718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:35.473539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1514963d92436eae7cf9617041b8dd', 'request_id': 'req-0e204519-ec4e-434b-95ff-a971e49999e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.549106 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1514963d92436eae7cf9617041b8dd', 'request_id': 'req-0e204519-ec4e-434b-95ff-a971e49999e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.555998 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1514963d92436eae7cf9617041b8dd', 'request_id': 'req-0e204519-ec4e-434b-95ff-a971e49999e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.567652 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] The token cache is being invalidate because project 3b1514963d92436eae7cf9617041b8dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:35.567939 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.568580 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.570150 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1428/2873] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:35 2022] DELETE /identity/v3/projects/3b1514963d92436eae7cf9617041b8dd => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:35.574067 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:35.575572 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:35.589242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58f0db-7784-4189-aadd-2decc9a98c1a admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:35.647771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:35.648645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:35.648964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:35.649242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] PATH_INFO: `/v3/users/32f349967335404bba261da2eb6f3cba` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:35.650437 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] RBAC: Authorizing `identity:delete_user(user_id=32f349967335404bba261da2eb6f3cba)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:35.652432 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:35.676351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:35.872720 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32f349967335404bba261da2eb6f3cba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.884963 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32f349967335404bba261da2eb6f3cba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.966508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32f349967335404bba261da2eb6f3cba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.974040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32f349967335404bba261da2eb6f3cba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.974700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-7b0f54f8-6b4d-4fcd-b11d-c4c8bdf9933e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32f349967335404bba261da2eb6f3cba'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:35.980838 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1446/2874] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:35 2022] DELETE /identity/v3/users/32f349967335404bba261da2eb6f3cba => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:36.236736 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:36.238218 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:36.251922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e204519-ec4e-434b-95ff-a971e49999e6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:36.312573 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:36.313510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.313784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.314053 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] PATH_INFO: `/v3/projects/9301e01a38654f1ea8bc2290304a710c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.315245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9301e01a38654f1ea8bc2290304a710c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:36.317200 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:36.412833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:36.426312 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9301e01a38654f1ea8bc2290304a710c', 'request_id': 'req-7436fdbc-61d5-4027-a7a2-9f5134de6251'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:36.526500 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9301e01a38654f1ea8bc2290304a710c', 'request_id': 'req-7436fdbc-61d5-4027-a7a2-9f5134de6251'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:36.534115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9301e01a38654f1ea8bc2290304a710c', 'request_id': 'req-7436fdbc-61d5-4027-a7a2-9f5134de6251'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:36.545349 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] The token cache is being invalidate because project 9301e01a38654f1ea8bc2290304a710c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:36.546233 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:36.546897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7436fdbc-61d5-4027-a7a2-9f5134de6251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:36.548741 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1429/2875] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:36 2022] DELETE /identity/v3/projects/9301e01a38654f1ea8bc2290304a710c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:36.553568 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.553854 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.554131 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.601399 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:36.602635 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:36.662468 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1447/2876] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:36.670230 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.671142 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.671428 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.690603 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:36.691841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:36.699782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1430/2877] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:36.704579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:36.714777 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa996f69-49cd-41ec-bad6-3f43f0787aef None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:36.720010 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:36.720733 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.721016 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.721286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.722000 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:34:36.722308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:36.723939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:36.739356 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1448/2878] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:34:36.740415 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:36.750227 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72ba5f-a541-466d-896d-450c18147806 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:36.754922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:36.755629 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.755912 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.756185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.756879 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:36.758809 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:36.862687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1431/2879] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:36 2022] POST /identity/v3/projects => generated 361 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:34:36.866766 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:36.875002 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e161f9-6d9b-4127-8cdd-163f213c832f admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:36.937877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:36.938573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:36.938860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:36.939129 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:36.940438 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:36.941906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:36.949771 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:37.058677 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1449/2880] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:36 2022] POST /identity/v3/users => generated 281 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:34:37.063944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:37.074157 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c48c5-5e60-4898-8bc0-16dd25a162a4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:37.079944 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:37.080616 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.080899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.081173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.081936 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:37.083605 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:37.094562 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1432/2881] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:37.098754 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:37.109345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7fb1b-6580-4971-b053-d029c48dac9b admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:37.114103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:37.114802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.115088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.115366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.116093 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:37.117699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4f963-bca4-4824-ba1b-48125c1be01f admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:37.129871 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1450/2882] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:37.133589 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:37.144413 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874025ac-47fb-485c-bc2b-94ef64d69ce5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:37.149253 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:37.150138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.150429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.150775 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] PATH_INFO: `/v3/projects/2c47e43a42604549a417c775b3b24509/users/5f63e4ffe02c406aa623b55819bb8116/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.167126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c47e43a42604549a417c775b3b24509, user_id=5f63e4ffe02c406aa623b55819bb8116, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:37.169856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:37.258073 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1433/2883] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:37 2022] PUT /identity/v3/projects/2c47e43a42604549a417c775b3b24509/users/5f63e4ffe02c406aa623b55819bb8116/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:37.270054 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.270345 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.270663 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.296859 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] MFA Rules not processed for user `5f63e4ffe02c406aa623b55819bb8116`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:37.298103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:37.365423 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1451/2884] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:37.399631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:37.401374 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:37.459075 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b3efa-2574-46ec-b2b1-8c0a31e34415 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:37.462794 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:37.463439 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.463719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.463995 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.472430 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:37.473861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d34e-8cc5-439b-b44a-e0a4ad239a5a service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:37.482560 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1434/2885] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:37.496633 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:37.560550 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a6403-db09-4b4e-a8c6-e7e9c0d2d601 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:37.565942 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:37.566891 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.567177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.567454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] PATH_INFO: `/v3/users/5f63e4ffe02c406aa623b55819bb8116` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.568683 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f63e4ffe02c406aa623b55819bb8116)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:37.570618 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:37.869619 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f63e4ffe02c406aa623b55819bb8116'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:37.882351 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f63e4ffe02c406aa623b55819bb8116'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:37.948044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f63e4ffe02c406aa623b55819bb8116'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:37.949596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:37.959853 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f63e4ffe02c406aa623b55819bb8116'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:37.960880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f63e4ffe02c406aa623b55819bb8116'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:37.967995 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1452/2886] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:34:37 2022] DELETE /identity/v3/users/5f63e4ffe02c406aa623b55819bb8116 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:37.972574 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:37.972860 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:37.973141 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:37.991914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:37.993128 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:38.043249 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:38.053126 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1435/2887] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:38.063045 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:38.064523 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:38.121877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c150-3606-498b-8ece-dd2943eb2ed6 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:38.125680 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:38.126327 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:38.126645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:38.126920 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:38.135989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:38.137427 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09875ec-880c-4182-aaf7-4422bbef36ed service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:38.148065 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1453/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:38.613373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:38.614878 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:38.627138 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e69d1-91fe-4abf-aa14-03723f358d5a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:38.633289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:38.633289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:38.633289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:38.633289 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] PATH_INFO: `/v3/projects/2c47e43a42604549a417c775b3b24509` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:38.634406 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c47e43a42604549a417c775b3b24509)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:38.636813 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:38.784083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c47e43a42604549a417c775b3b24509', 'request_id': 'req-60c80fa9-7790-4a08-b3fa-c1324ae7a914'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:38.863530 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c47e43a42604549a417c775b3b24509', 'request_id': 'req-60c80fa9-7790-4a08-b3fa-c1324ae7a914'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:38.870322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c47e43a42604549a417c775b3b24509', 'request_id': 'req-60c80fa9-7790-4a08-b3fa-c1324ae7a914'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:38.881517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] The token cache is being invalidate because project 2c47e43a42604549a417c775b3b24509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:34:38.881802 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:38.882433 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60c80fa9-7790-4a08-b3fa-c1324ae7a914 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:34:38.884104 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1436/2889] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:34:38 2022] DELETE /identity/v3/projects/2c47e43a42604549a417c775b3b24509 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:38.888964 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:38.890114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:38.890114 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:38.907735 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:38.908866 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:38.968319 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1454/2890] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:38.976054 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:38.976343 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:38.976621 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:38.995381 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:38.996587 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:39.005165 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1437/2891] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:39.010293 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.019982 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea9a3b-15c9-4bbb-8754-b0b943c54ffe None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.024506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.025144 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.025422 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.025694 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.026410 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:34:39.026743 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.028333 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.039707 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1455/2892] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:34:39.043733 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.054015 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab950bde-0940-460a-a53e-0d85c331c799 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.060011 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.060682 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.060963 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.061922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.062662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.064557 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.150678 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1438/2893] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:34:39 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:39.155481 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.168347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c993d7-5c58-4016-a6ce-37411004b4fd admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.223230 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.223896 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.224181 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.224457 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.225148 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.227103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.330901 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1456/2894] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:34:39 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:34:39.334915 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.345841 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02907f4-b019-4ea1-a984-508276497d6f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.350474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.351151 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.351429 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.351701 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.352421 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.354017 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.365302 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1439/2895] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:39.369307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.379323 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebd11a-7c12-4150-ba41-8b227d80a553 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.383989 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.384642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.384926 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.385224 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.385940 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.387584 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711cfa0-176d-4e76-8860-f48535f6d83c admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.389603 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:39.398883 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1457/2896] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:34:39.403179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.414651 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5195c3-205d-4e2d-8955-1787ab00b2de admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.421326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.421326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.421326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.421612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] PATH_INFO: `/v3/projects/b2c26e5d41364cc580f4d0389c8fa728/users/f78ec5810d724af0985d2d13aa549124/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.437496 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c26e5d41364cc580f4d0389c8fa728, user_id=f78ec5810d724af0985d2d13aa549124, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.440245 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.534904 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1440/2897] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:34:39 2022] PUT /identity/v3/projects/b2c26e5d41364cc580f4d0389c8fa728/users/f78ec5810d724af0985d2d13aa549124/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:34:39.540033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af400-650c-4058-a11b-358a63224e8d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.540322 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af400-650c-4058-a11b-358a63224e8d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.540597 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2af400-650c-4058-a11b-358a63224e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.558756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-7d2af400-650c-4058-a11b-358a63224e8d None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:39.560088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-7d2af400-650c-4058-a11b-358a63224e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:39.620392 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1458/2898] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:39.630590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:39.632322 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:39.691568 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfc4b5-d5c5-4b88-943f-2a3dd2fa3082 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:39.695485 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:39.696162 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:39.696437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:39.696790 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:39.706716 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:39.708156 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:39.718667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1441/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:41.013373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b69db-8a7d-4053-bce2-b66c8a1a0539 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:41.019562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b69db-8a7d-4053-bce2-b66c8a1a0539 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:41.019562 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b69db-8a7d-4053-bce2-b66c8a1a0539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:41.045494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-f43b69db-8a7d-4053-bce2-b66c8a1a0539 None None] MFA Rules not processed for user `f78ec5810d724af0985d2d13aa549124`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:41.047486 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-f43b69db-8a7d-4053-bce2-b66c8a1a0539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:41.125578 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1459/2900] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:41.136083 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:41.137662 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:41.167153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:41.199991 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d245d7-6e55-4407-9741-878ce366b68c service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:41.204447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c42ba8788bf2431a96b2087b287d7426', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:41.205192 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:41.205883 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:41.206325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:41.215467 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:41.216953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:41.226509 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1442/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:43.002009 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:43.002540 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:43.002653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:43.020544 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] MFA Rules not processed for user `f78ec5810d724af0985d2d13aa549124`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:43.021767 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:43.028834 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1460/2902] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:34:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:34:43.068443 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:43.069830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:43.129943 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292fefc-dff3-4c87-ab34-526b4c6e76f9 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:43.133763 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:43.134465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:43.134784 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:43.135082 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:43.136868 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:43.228085 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:43.229449 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47aecb1-8a08-4727-a5e1-7c75e30bf2d8 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:43.249861 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1443/2903] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:43.454704 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:43.456130 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:43.468788 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31051d7f-c621-4013-baf3-0906502e3ec3 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:43.472414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:43.473044 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:43.473321 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:43.473592 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:43.475350 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:43.486069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:43.486069 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:43.494161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1461/2904] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:43.951402 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:43.952005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:43.953881 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:43.969653 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:43.982840 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf61df7d-8b47-4b19-aa68-efed2a4c60be service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:43.991986 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:43.992752 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:43.993064 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:43.993334 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:43.995264 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:44.009788 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:34:44.011220 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:44.012352 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:44.013880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:44.028416 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1462/2905] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:34:44.036730 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:44.083329 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1444/2906] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:34:44.095602 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:44.097025 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:44.112517 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6cb8e-ade7-43f3-92db-64c838cc39a6 service neutron] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:44.118139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:44.118852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:44.119113 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:44.119505 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:44.129974 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:44.131675 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:44.143059 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1463/2907] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:47.393723 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:47.403465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65fc1c-a85c-45a8-a1c9-686ea9179e08 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:47.407972 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:47.408626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:47.408910 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:47.409185 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:47.411136 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:47.421498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:47.423115 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351bec97-fe8c-4826-a7bb-03a46cb8345e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:47.432059 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1445/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:54.137668 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=81 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:34:56.915886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:34:56.917530 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:56.933897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac896b-202e-4764-9fef-3000907c7484 service placement] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:34:56.940177 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:34:56.940827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:34:56.941109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:34:56.941442 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:34:56.943253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:34:56.957193 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:34:56.958852 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b777d8-6001-4233-9b16-ab238fe9a688 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:34:56.970698 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1464/2909] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:34:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:34:57.428039 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=23 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:05.164828 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:05.165437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:05.165437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:05.184040 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] MFA Rules not processed for user `f78ec5810d724af0985d2d13aa549124`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:05.185336 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:05.193236 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1446/2910] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:06.823802 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54fe7-d526-4d36-9c3d-2a240caa8340 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:06.825691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54fe7-d526-4d36-9c3d-2a240caa8340 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:06.825691 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54fe7-d526-4d36-9c3d-2a240caa8340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:06.843778 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-5af54fe7-d526-4d36-9c3d-2a240caa8340 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:06.845103 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-5af54fe7-d526-4d36-9c3d-2a240caa8340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:06.854227 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1465/2911] 10.222.0.23 () {58 vars in 1018 bytes} [Fri Jun 24 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:06.866225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:06.868234 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:06.886613 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2831fe7-f539-44d2-9472-5d6168b96cb1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:06.892727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:06.893389 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:06.893672 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:06.893947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:06.903224 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:06.904844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43751bb-6376-4139-a312-4fe0f7ec56a6 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:06.915817 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1447/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:16.838414 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=9 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:16.911320 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=9 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:36.134730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.135231 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.135328 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.185415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:36.186649 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:36.250467 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1466/2913] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:36.375856 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad9d9-e0ef-4ed3-bace-bb513125ec91 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.376347 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad9d9-e0ef-4ed3-bace-bb513125ec91 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.376442 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ad9d9-e0ef-4ed3-bace-bb513125ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.426113 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-875ad9d9-e0ef-4ed3-bace-bb513125ec91 None None] MFA Rules not processed for user `52486f194961487ba574eebe49869a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:36.427386 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-875ad9d9-e0ef-4ed3-bace-bb513125ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:36.499508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1448/2914] 10.222.0.23 () {58 vars in 1039 bytes} [Fri Jun 24 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:36.505702 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:36.518316 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4a196-0e60-4072-b674-51c4ee48e96a None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:36.524008 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:36.524594 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:36.524878 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:36.525156 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:36.533524 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:36.535150 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:36.543508 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1467/2915] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 15:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:35:46.496601 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=44 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:46.540208 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=46 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:52.806844 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6611-2017-49af-a29f-7985ad083213 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:52.807327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6611-2017-49af-a29f-7985ad083213 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:52.807447 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e6611-2017-49af-a29f-7985ad083213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:52.833812 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-895e6611-2017-49af-a29f-7985ad083213 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:52.835056 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-895e6611-2017-49af-a29f-7985ad083213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:52.842622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1449/2916] 10.222.0.23 () {58 vars in 967 bytes} [Fri Jun 24 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:52.856197 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:52.880040 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6b7d6-1dcd-4df7-a0e7-a38dffc72849 service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:52.885268 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:52.885906 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:52.886187 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:52.886461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:52.894699 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:52.896645 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:52.906376 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1468/2917] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 15:35:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:35:55.747003 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6611-2017-49af-a29f-7985ad083213 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:55.759830 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e6611-2017-49af-a29f-7985ad083213 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:55.764980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:55.765681 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:55.765954 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:55.766229 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:55.768086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:55.780051 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:55.781644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:55.791791 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1450/2918] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:35:55.870911 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:55.881600 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f1151-3244-41e5-94e0-8ccae55a704f service swift] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:55.887424 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:55.888289 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:55.888573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:55.888849 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] PATH_INFO: `/v3/users/f78ec5810d724af0985d2d13aa549124` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:55.890029 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78ec5810d724af0985d2d13aa549124)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:55.892075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.031243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78ec5810d724af0985d2d13aa549124'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.043409 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78ec5810d724af0985d2d13aa549124'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.097875 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78ec5810d724af0985d2d13aa549124'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.105705 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78ec5810d724af0985d2d13aa549124'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.106393 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-23623372-234e-4b02-97e1-e6d48038ff16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78ec5810d724af0985d2d13aa549124'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.112296 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1469/2919] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:35:55 2022] DELETE /identity/v3/users/f78ec5810d724af0985d2d13aa549124 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:56.345867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.357517 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a504baf-2ac7-49c3-94fb-c2a5cfcae5ac service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.420309 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.421217 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.421498 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.421773 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] PATH_INFO: `/v3/projects/b2c26e5d41364cc580f4d0389c8fa728` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.423013 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c26e5d41364cc580f4d0389c8fa728)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.425173 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.504048 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c26e5d41364cc580f4d0389c8fa728', 'request_id': 'req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.572137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c26e5d41364cc580f4d0389c8fa728', 'request_id': 'req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.578657 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c26e5d41364cc580f4d0389c8fa728', 'request_id': 'req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.590510 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] The token cache is being invalidate because project b2c26e5d41364cc580f4d0389c8fa728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:35:56.590770 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.591404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7ea97a09-cfe1-4c60-9401-88ecf4f2c963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:35:56.592946 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1451/2920] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:35:56 2022] DELETE /identity/v3/projects/b2c26e5d41364cc580f4d0389c8fa728 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:35:56.599152 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.599439 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.599740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.617617 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:56.618881 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:56.676933 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1470/2921] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:35:56.684808 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.685098 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.685378 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.703615 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:56.704801 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:56.713057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1452/2922] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:35:56.717253 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.727018 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c92e1b-162a-4187-be0d-ec5bd6ee3370 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.732251 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.732905 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.733254 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.733471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.734199 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80353) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 15:35:56.734506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.736205 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.748768 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1471/2923] 10.222.0.23 () {58 vars in 1152 bytes} [Fri Jun 24 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 15:35:56.753174 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.763355 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76734f-a0ac-4724-a76f-b0b2f04e8230 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.768074 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.768725 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.769009 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.769288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.769981 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.771941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:56.865654 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1453/2924] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:56.869606 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:56.879820 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8e268-275d-4e4c-8488-0959099fb7f7 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:56.911506 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:56.938471 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:56.939174 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:56.939454 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:56.939729 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:56.940419 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:56.942446 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.036178 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1472/2925] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:35:56 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:35:57.040417 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.051290 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4ba7c-ba79-44d5-aab5-ed0e033b5b18 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.056005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.056718 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.057012 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.057291 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.058026 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.059749 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.071978 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1454/2926] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:57.075954 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.085960 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741814ba-a453-4a48-b677-2f974705b771 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.091742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.092430 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.092719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.092999 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.093722 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.095354 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da802014-34f2-4bb3-8eaa-8fa2d2d01740 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.105762 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1473/2927] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:35:57.109692 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.120641 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d68a3-a7c3-4f22-a9e8-72f8e19d93e4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.125288 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.126153 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.126425 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.126732 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] PATH_INFO: `/v3/projects/4a609ad3492d468888849b908617b776/users/38a9a5a583de4421bc1337035708e871/roles/66ef983ebbae442b8cab404599a2d333` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.142609 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a609ad3492d468888849b908617b776, user_id=38a9a5a583de4421bc1337035708e871, role_id=66ef983ebbae442b8cab404599a2d333)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.145261 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.195908 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:35:57.202121 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1455/2928] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:35:57 2022] PUT /identity/v3/projects/4a609ad3492d468888849b908617b776/users/38a9a5a583de4421bc1337035708e871/roles/66ef983ebbae442b8cab404599a2d333 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 15:35:57.214979 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb681-7d16-4a12-9f12-6f480a235208 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.215269 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb681-7d16-4a12-9f12-6f480a235208 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.215573 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb681-7d16-4a12-9f12-6f480a235208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.233860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-1fedb681-7d16-4a12-9f12-6f480a235208 None None] MFA Rules not processed for user `ecb45340b1984b23904f9e0f1752bcc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:35:57.235107 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-1fedb681-7d16-4a12-9f12-6f480a235208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:57.292887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1474/2929] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:35:57.302833 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:35:57.304611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:35:57.363176 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994f0e3-7db9-404c-96e4-4e0102f2e151 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:35:57.366983 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:35:57.367631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:35:57.368078 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:35:57.368372 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:35:57.377622 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:35:57.379137 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:35:57.388611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1456/2930] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:02.652542 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:02.653075 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:02.653186 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:02.690827 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] MFA Rules not processed for user `38a9a5a583de4421bc1337035708e871`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:02.692092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:02.770507 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1475/2931] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:02.775863 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:02.786014 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e48acff-784a-477f-9621-f88f082821fb service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:02.795867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:02.795867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:02.795867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:02.795867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:02.796687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:02.796687 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:02.906941 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1457/2932] 10.222.0.23 () {60 vars in 1152 bytes} [Fri Jun 24 15:36:02 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 15:36:02.910814 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:02.930195 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a1c88-9e7d-4d36-a110-4a29c6bb631b None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:02.972440 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:02.989088 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:02.989755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:02.990037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:02.990308 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] PATH_INFO: `/v3/users` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:02.991017 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:02.992950 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:03.089046 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1476/2933] 10.222.0.23 () {60 vars in 1143 bytes} [Fri Jun 24 15:36:02 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 15:36:03.092501 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:03.103327 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff82f48-e51a-4215-a0fe-bbcd88b51c24 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:03.108205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:03.108974 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:03.109256 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:03.109529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:03.110285 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:03.111947 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:03.128930 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1458/2934] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:36:03.129039 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:03.139764 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401174-1c52-430e-b2df-689bbb9dddaf admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:03.145185 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:03.146078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:03.146366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:03.146692 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] PATH_INFO: `/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4/users/218386e9b62c43578482c4d911e77fca/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:03.167037 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1c0f416842490495d5c3e8e2c91bd4, user_id=218386e9b62c43578482c4d911e77fca, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:03.169719 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:03.270742 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1477/2935] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:36:03 2022] PUT /identity/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4/users/218386e9b62c43578482c4d911e77fca/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:03.275444 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:03.285474 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4e5db-fec4-4576-8468-b27fa528c52c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:03.342385 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:03.343091 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:03.343373 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:03.343649 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:03.344377 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:03.345984 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518a824-1136-4b0f-adc3-536b030ae4cd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:03.356922 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1459/2936] 10.222.0.23 () {58 vars in 1121 bytes} [Fri Jun 24 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 15:36:03.360887 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:03.371358 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c67892-27f9-4c33-84fd-bb2965e2bded admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:03.375983 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:03.376860 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:03.377140 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:03.377415 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] PATH_INFO: `/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4/users/218386e9b62c43578482c4d911e77fca/roles/4ef72abf5489405bbc82b2318479765b` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:03.379513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1c0f416842490495d5c3e8e2c91bd4, user_id=218386e9b62c43578482c4d911e77fca, role_id=4ef72abf5489405bbc82b2318479765b)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:03.382070 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ddbd-d35d-42b3-abd4-f64e61854d91 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:03.393998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1478/2937] 10.222.0.23 () {60 vars in 1482 bytes} [Fri Jun 24 15:36:03 2022] PUT /identity/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4/users/218386e9b62c43578482c4d911e77fca/roles/4ef72abf5489405bbc82b2318479765b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:36:08.904719 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:08.905259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:08.905259 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:09.085626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] MFA Rules not processed for user `218386e9b62c43578482c4d911e77fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:09.085626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:09.085626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:09.085626 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1460/2938] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:09.089206 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635926-9fd5-4ccc-aed8-afbbc7014930 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:09.091099 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635926-9fd5-4ccc-aed8-afbbc7014930 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:09.091382 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635926-9fd5-4ccc-aed8-afbbc7014930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:09.156700 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-ea635926-9fd5-4ccc-aed8-afbbc7014930 None None] MFA Rules not processed for user `bb628c1d9c854cbca433d36d143374c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:09.158373 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-ea635926-9fd5-4ccc-aed8-afbbc7014930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:09.227299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1479/2939] 10.222.0.23 () {58 vars in 1059 bytes} [Fri Jun 24 15:36:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:09.234155 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:09.244720 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dfe55-28d4-4d2e-8b57-c9df3814ef09 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:09.248331 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb628c1d9c854cbca433d36d143374c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:09.248980 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:09.249264 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:09.249534 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:09.315225 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:09.316666 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:09.325793 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1461/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:09.557256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120729-531a-4499-9dda-4def47086ba9 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:09.557648 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120729-531a-4499-9dda-4def47086ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:09.557921 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e120729-531a-4499-9dda-4def47086ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:09.577015 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-6e120729-531a-4499-9dda-4def47086ba9 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:09.580108 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-6e120729-531a-4499-9dda-4def47086ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:09.634534 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1480/2941] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 15:36:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:09.646317 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:09.647835 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:09.721824 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25ab4a-2687-43b2-bc18-f16138a74dac service glance] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:09.727611 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:09.728825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:09.729110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:09.729391 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:09.738472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:09.740179 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:09.748644 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1462/2942] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:19.626888 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:19.745341 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=65 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:20.377068 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62378fe-56a3-4075-83a1-f99bfa5bd596 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:20.377353 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62378fe-56a3-4075-83a1-f99bfa5bd596 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:20.377629 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62378fe-56a3-4075-83a1-f99bfa5bd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:20.396998 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-b62378fe-56a3-4075-83a1-f99bfa5bd596 None None] MFA Rules not processed for user `38a9a5a583de4421bc1337035708e871`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:20.398212 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-b62378fe-56a3-4075-83a1-f99bfa5bd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:20.405988 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1481/2943] 10.222.0.23 () {58 vars in 957 bytes} [Fri Jun 24 15:36:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:20.653390 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:20.710111 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e4a83-d531-45cc-9048-75ef732cb757 service swift] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:20.714164 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:20.714867 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:20.715149 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:20.715420 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:20.717206 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:20.726727 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:20.731539 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:20.736744 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1463/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:24.155100 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:24.155589 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:24.157461 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:24.174755 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:36:24.175973 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:24.183642 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1482/2945] 10.222.0.23 () {58 vars in 1009 bytes} [Fri Jun 24 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:36:24.201899 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:24.214969 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7605c4-331a-4331-bd05-a9ad223d1d23 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:24.220099 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:24.220750 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:24.221036 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:24.221323 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:24.230914 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:24.232525 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:24.241260 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1464/2946] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 15:36:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:36:34.169494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:34.237638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:37.268204 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:37.270317 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:37.361329 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf5f8-c61d-41ce-9444-aa3a01d85443 None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:37.367092 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:37.367763 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:37.368052 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:37.368338 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:37.370728 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:36:37.385730 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:37.392288 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:37.400678 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1483/2947] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:36:47.394170 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=34 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:36:58.725237 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:36:58.735535 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e466dce-2993-4b36-90e3-7f1eacbd7bb0 service swift] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:36:58.739228 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:36:58.739897 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:36:58.740180 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:36:58.740460 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:36:58.750953 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:36:58.752394 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:36:58.763068 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1465/2948] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:36:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:37:00.692494 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:37:00.693939 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:37:00.709727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f33d63-9052-4042-81ed-ae9c94bd0dd9 service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:37:00.716307 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4bfc33870df145a4ba481afd9cdab92b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:37:00.716957 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:37:00.717242 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:37:00.717513 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:37:00.719300 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:37:00.733797 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:37:00.735464 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:37:00.746897 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1484/2949] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:37:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:37:08.757602 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:37:10.742109 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=6 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:38:12.556490 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:38:12.558057 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:38:12.657005 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2b830-eeb2-4f63-87cc-fbe063cfe1c9 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:38:12.662125 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:38:12.662797 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:38:12.663084 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:38:12.663358 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:38:12.665688 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:38:12.677667 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:38:12.679306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:38:12.690271 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1466/2950] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:38:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:38:22.685892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=44 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:39:43.631492 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:43.651299 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0bb88-c594-41a5-95da-bf48f32e90bb service nova] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:43.654948 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:43.655596 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] REQUEST_METHOD: `GET` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:43.655880 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:43.656161 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:43.657944 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:43.682817 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:43.684243 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3122-5865-45c0-8b4f-2ff40a15ecbd service neutron] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:43.695366 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1485/2951] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:39:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:39:44.868105 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:44.887989 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acbe7-5682-4a56-9f71-ee75a424d290 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:44.891631 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd08dcd615d87450d9ccdc58fe89f4014', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:44.892272 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:44.892555 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:44.892834 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:44.895342 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:44.908696 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:44.910396 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:44.921392 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1467/2952] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:39:46.912739 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887eb38-743e-4140-8e6c-3899d8f98d5d None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:46.913280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887eb38-743e-4140-8e6c-3899d8f98d5d None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:46.913280 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887eb38-743e-4140-8e6c-3899d8f98d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:46.942825 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-9887eb38-743e-4140-8e6c-3899d8f98d5d None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:39:46.944033 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-9887eb38-743e-4140-8e6c-3899d8f98d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:46.953078 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1486/2953] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 15:39:46.965298 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:46.977437 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014309c-040d-44c5-b467-5c85a32dfcaa service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:46.982561 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:46.983184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:46.983465 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:46.983742 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:46.992917 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:46.995934 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:47.004638 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1468/2954] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 15:39:48.513456 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06bdae-58e2-408f-afed-e8a9f61d7875 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:48.517278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06bdae-58e2-408f-afed-e8a9f61d7875 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:48.517278 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06bdae-58e2-408f-afed-e8a9f61d7875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:48.531546 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-dd06bdae-58e2-408f-afed-e8a9f61d7875 None None] MFA Rules not processed for user `8f0210144ae443e8a6d1b9e0fe196984`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:39:48.532756 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-dd06bdae-58e2-408f-afed-e8a9f61d7875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:48.540502 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1487/2955] 10.222.0.23 () {58 vars in 967 bytes} [Fri Jun 24 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:39:48.581133 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:48.593299 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a44507-152b-485b-9cbe-fd61558262a6 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:48.599110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52486f194961487ba574eebe49869a96', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:48.599753 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:48.600033 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:48.600306 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:48.608612 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:48.610184 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:48.618832 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1469/2956] 10.222.0.23 () {60 vars in 1384 bytes} [Fri Jun 24 15:39:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 15:39:53.072537 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:53.077275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:53.077275 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:53.092073 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:39:53.093325 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:53.102877 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1488/2957] 10.222.0.23 () {58 vars in 1057 bytes} [Fri Jun 24 15:39:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:39:53.109046 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:53.120559 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca6897-dfee-4667-9c1c-3e73df01e97b service swift] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:53.125635 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:53.126277 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:53.126590 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:53.126871 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:53.129232 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:53.140768 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:53.142325 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:53.157404 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1470/2958] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:39:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:39:56.845992 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:56.917886 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322df9bc-f68b-4646-8a29-592a9736f35c None None] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:56.922579 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:56.923448 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:56.923727 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:56.924001 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] PATH_INFO: `/v3/users/38a9a5a583de4421bc1337035708e871` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:56.925936 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a9a5a583de4421bc1337035708e871)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:56.928043 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:57.129943 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a9a5a583de4421bc1337035708e871'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.141947 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a9a5a583de4421bc1337035708e871'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.187749 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a9a5a583de4421bc1337035708e871'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.195718 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a9a5a583de4421bc1337035708e871'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.196347 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a9a5a583de4421bc1337035708e871'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.202261 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1489/2959] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:39:56 2022] DELETE /identity/v3/users/38a9a5a583de4421bc1337035708e871 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:39:57.447086 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:57.452472 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a31d8-67ff-475b-bfae-e4c19e6a9f58 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:57.512110 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:57.512964 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:57.513240 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:57.513508 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] PATH_INFO: `/v3/projects/4a609ad3492d468888849b908617b776` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:57.514724 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a609ad3492d468888849b908617b776)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:57.516627 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:57.596996 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a609ad3492d468888849b908617b776', 'request_id': 'req-681decc3-bbff-4828-a0bd-6ef6b67fa694'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.655205 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a609ad3492d468888849b908617b776', 'request_id': 'req-681decc3-bbff-4828-a0bd-6ef6b67fa694'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.662002 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a609ad3492d468888849b908617b776', 'request_id': 'req-681decc3-bbff-4828-a0bd-6ef6b67fa694'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.672529 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] The token cache is being invalidate because project 4a609ad3492d468888849b908617b776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:39:57.672845 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.673483 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:57.675905 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1471/2960] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:39:57 2022] DELETE /identity/v3/projects/4a609ad3492d468888849b908617b776 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:39:57.680011 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Authenticating user token {{(pid=80353) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:57.681451 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:57.694482 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0eb52-8d0d-4f4d-9224-facd58d7a000 admin admin] Validating token access rules against request {{(pid=80353) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:57.706387 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:39:57.751200 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80353) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:57.752116 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:57.752640 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:57.752912 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] PATH_INFO: `/v3/users/218386e9b62c43578482c4d911e77fca` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:57.754183 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] RBAC: Authorizing `identity:delete_user(user_id=218386e9b62c43578482c4d911e77fca)` {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:57.756256 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] RBAC: Authorization granted {{(pid=80353) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:57.991684 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218386e9b62c43578482c4d911e77fca'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.004581 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218386e9b62c43578482c4d911e77fca'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.068498 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218386e9b62c43578482c4d911e77fca'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.075904 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218386e9b62c43578482c4d911e77fca'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.080488 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.notifications [None req-c55f827d-76e8-4425-8a2d-b82d5842713e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218386e9b62c43578482c4d911e77fca'} {{(pid=80353) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.083459 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1490/2961] 10.222.0.23 () {58 vars in 1223 bytes} [Fri Jun 24 15:39:57 2022] DELETE /identity/v3/users/218386e9b62c43578482c4d911e77fca => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:39:58.342913 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:39:58.344364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:39:58.357388 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681decc3-bbff-4828-a0bd-6ef6b67fa694 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:39:58.385100 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:39:58.417189 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecb45340b1984b23904f9e0f1752bcc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7486fcd7991c440786c9d72536ea0aed', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:39:58.418088 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:39:58.418370 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:39:58.418670 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] PATH_INFO: `/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:39:58.419886 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1c0f416842490495d5c3e8e2c91bd4)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:39:58.421810 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:39:58.541367 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1c0f416842490495d5c3e8e2c91bd4', 'request_id': 'req-bd4f8b88-439a-4b95-9171-fa6c39a73b73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.615375 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1c0f416842490495d5c3e8e2c91bd4', 'request_id': 'req-bd4f8b88-439a-4b95-9171-fa6c39a73b73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.622035 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1c0f416842490495d5c3e8e2c91bd4', 'request_id': 'req-bd4f8b88-439a-4b95-9171-fa6c39a73b73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.633047 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] The token cache is being invalidate because project bf1c0f416842490495d5c3e8e2c91bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 15:39:58.633337 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.633988 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 15:39:58.635577 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1472/2962] 10.222.0.23 () {58 vars in 1232 bytes} [Fri Jun 24 15:39:58 2022] DELETE /identity/v3/projects/bf1c0f416842490495d5c3e8e2c91bd4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 15:40:08.078946 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:40:08.630213 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:45:23.568740 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edf0dd-2edf-440c-b000-34a8a65ea748 None None] REQUEST_METHOD: `POST` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:45:23.569192 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edf0dd-2edf-440c-b000-34a8a65ea748 None None] SCRIPT_NAME: `/identity` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:45:23.569294 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edf0dd-2edf-440c-b000-34a8a65ea748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80353) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:45:23.615087 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.auth.core [None req-c8edf0dd-2edf-440c-b000-34a8a65ea748 None None] MFA Rules not processed for user `812d75984e6245e2939c28fbd068b895`. Rule list: `[]` (Enabled: `True`). {{(pid=80353) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 15:45:23.616286 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG keystone.common.fernet_utils [None req-c8edf0dd-2edf-440c-b000-34a8a65ea748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80353) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:45:23.677846 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: [pid: 80353|app: 0|req: 1491/2963] 10.222.0.23 () {58 vars in 1057 bytes} [Fri Jun 24 15:45:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 15:45:23.683993 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 15:45:23.695782 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8b88-439a-4b95-9171-fa6c39a73b73 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 15:45:23.700693 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812d75984e6245e2939c28fbd068b895', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3ad4c7896fe34e77910020886bf34d03', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 15:45:23.701326 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 15:45:23.701633 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 15:45:23.701906 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 15:45:23.703825 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 15:45:23.715783 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 15:45:23.717364 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5423d50-3f5a-483d-83ef-1d45209d11e3 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 15:45:23.727593 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1473/2964] 10.222.0.23 () {60 vars in 1365 bytes} [Fri Jun 24 15:45:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 15:45:33.669139 dv-cinder-844601-5-iscsi devstack@keystone.service[80353]: DEBUG dbcounter [-] [80353] Writing DB stats keystone:SELECT=39 {{(pid=80353) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 15:45:33.722892 dv-cinder-844601-5-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}